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R-2607-0NR  Improving  Training  and  Per¬ 
formance  of  Navy  Teams:  A  Design  for  a 
Research  Program.)!/  P.  W.  Thorndyke, 

M.  G.  Weiner,  P.  Carpenter-Huffman, 

T.  Donaldson,  R.  S.  Gaines,  B. 
Hayes-Roth,  A.  A.  Lewis,  M.  Menchik,  S. 
Taylor,  K.  T.  Wescourt.  July  1980. 

A  design  for  a  large-scale  research  program 
on  improving  Navy  team  effectiveness.  The 
report  assesses  the  most  critical  Navy 
teams,  identifies  deficiencies  in  the  per¬ 
formance  of  these  teams,  and  recommends  new 
research  that  could  lead  to  significant  im- 
rov^ments  in  team  performance.  The  major 
conclusions  are  (1)  teams  performing 
time-stressed  decisionmaking  using  symbolic 
information  are  most  critical  to  mission 
effectiveness  and  ship  survivability,  (2) 
several  currently  available  instructional 
methods  could  immediately  improve  training, 
(3)  new  interdisciplinary  research  should 
study  simulated  teams  in  laboratory  task 
environments  that  provide  computer-control- 
led  task  scenarios,  realistic  environmental 
and  enemy  models,  and  intra-team  communica¬ 
tions  networks,  and  (4)  promising  ap¬ 
proaches  to  improving  teams  include  im¬ 
proving  performance  models  of  team  tasks, 
improving  the  tools  and  methods  of  train¬ 
ing,  compensating  for  disruptive  effects  of 
turnover  in  team  personnel,  improving  team 
organization,  and  improving  human-machine 
systems  to  aid  task  performance.  80  pp. 
(Author)  ^ 
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PREFACE 


In  a  1976  report  prepared  for  the  Secretary  of  Defense,  the  Defense  Science 
Board  Task  Force  on  Training  Technology  recommended  an  increase  in  funds  for 
research  and  development  in  training  technology.  As  an  indirect  result  of  that 
study,  the  Psychological  Sciences  Division  of  the  Office  of  Naval  Research  intends 
to  embark  on  a  program  of  research  in  the  area  of  Team  Training  and  Performance. 
Under  ONR  Contract  N00014-79-C-0753,  the  Rand  Team  Performance  Project  was 
formed  to  design  this  research  program  between  September  1, 1979,  and  February 
2&4980.  - 

Fourteen  Rand  staff  members  and  consultants  participated  in  the  project. 
Knowledge  of  prior  and  current  research  on  teams  and  research  techniques  of 
potential  utility  in  studying  teams  were  garnered  from  several  sources.  These 
included  a  review  of  the  literature  related  to  teams  (a  bibliography  of  this  literature 
appears  at  the  end  of  this  report),  the  compilation  of  a  library  of  books  and  docu¬ 
ments  on  teams,  and  the  Rand  Team  Performance  Workshop,  held  November  27-29, 
1979.  (The  proceedings  of  this  workshop  are  being  published  as  a  separate  Rand 
report.)  In  addition,  conversations  with  representatives  of  the  Military  Depart¬ 
ments  and  visits  to  Navy  training  and  research  installations  provided  insights  into 
the  procedures  and  problems  of  operational  Navy  teams.  Sites  visited  by  project 
personnel  included  the  Pacific  Fleet  Training  Command,  the  Fleet  ASW  Training 
Center,  the  Fleet  Combat  Training  Center,  the  Fleet  Combat  Systems  Training 
Unit,  and  the  Naval  Ocean  Systems  Center. 

This  report  presents  the  project  conclusions  and  recommendations  for  the  de¬ 
sign  and  content  of  the  research  program.  It  describes  a  large-scale  research  and 
development  effort  focusing  on  types  of  teams  represented  in  all  Service  branches. 
Thus,  the  report  should  interest  potential  contractors  in  the  research  and  develop¬ 
ment  communities  (including  policy  analysts,  psychologists,  computer  scientists, 
and  management  scientists),  program  management  personnel  in  the  Navy  and 
other  Services,  and  personnel  in  training  and  operational  commands. 
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i. 


Many  Navy  operations  depend  upon  the  coordinated  actions  of  a  team  of  indi¬ 
viduals.  Currently,  several  obstacles  hinder  the  formation,  training,  and  mainte¬ 
nance  of  high-quality  Navy  teams.  These  obstacles  include  restrictions  on  training 
time,  the  frequent  lack  of  training  objectives  and  performance  criteria,  increasing 
complexity  of  equipment  and  jobs,  declining  basic  skills  of  entering  personnel,  high 
rates  of  personnel  turnover,  and  frequent  failures  to  translate  successful  research 
results  into  improvements  in  the  operational  environment. 

Against  this  background,  the  Office  of  Naval  Research  (ONR)  charged  the  Rand 
Team  Performance  Project  with  the  task  of  designing  a  research  program  aimed 
at  the  improvement  of  Navy  team  effectiveness.  In  particular,  ONR  requested  that 
this  design  project  address  research  needs  in  the  areas  of  team  training,  develop¬ 
ment,  and  evaluation.  ONR  also  sought  recommendations  on  the  proper  roles  of 
different  research  disciplines  and  methodologies  in  the  program. 

The  major  recommendations  for  the  design  of  the  research  program  are  con¬ 
tained  in  a  set  of  eleven  conclusions.  These  conclusions  provide  a  supporting  frame¬ 
work  for  a  set  of  more  specific  recommendations  for  management  activities  to 
implement  the  program  and  research  projects  within  the  program.  The  major 
project  conclusions  follow;  each  contains  a  reference  to  that  section  of  the  report 
presenting  supporting  arguments  for  it. 

Conclusion  1:  The  teams  for  which  ONR-supported  research  is  likely  to  have 
the  greatest  impact  are  those  that  process  large  amounts  of  symbolic  information 
and  make  tactical  decisions  under  considerable  time  stress.  These  teams  are  the 
most  critical  for  combat  effectiveness  and  ship  survivability  and  are  ubiquitous  in 
the  Navy  and  in  the  other  Services.  Moreover,  their  effectiveness  depends  on  the 
coordination  and  organization  of  the  team  in  addition  to  the  skilled  performance 
of  individual  team  members  (see  Sections  2.1-2.4). 

Conclusion  2:  Current  personnel  problems  and  Navy  policies  limit  team  per¬ 
formance.  To  improve  Navy  teams,  research  should  identify  organizational  poli¬ 
cies,  technological  advances,  and  changes  in  team  training  and  procedures  that  will 
improve  the  proficiency  of  the  people  staffing  Navy  teams  in  the  future  (see  Sec¬ 
tions  1,  2.5,  and.  2.6). 

Conclusion  3:  Several  techniques  currently  exist  for  making  immediate  im¬ 
provements  in  the  training  of  team  members.  These  include  providing  objective 
feedback  during  training,  improving  the  match  between  training  and  job  require¬ 
ments,  and  expanding  the  use  of  computer-based  instruction.  Technology-transfer 
studies  should  be  undertaken  to  determine  whether  and  how  these  improvements 
should  be  implemented  in  operational  environments  (see  Sections  2.5  and  5.1.3). 

Conclusion  4:  New,  longer-term,  basic  research  should  include  the  study  of 
decisionmaking  teams  in  realistic  task  environments.  While  this  would  not  neces¬ 
sarily  require  continual  direct  contact  between  researchers  and  Navy  teams,  re¬ 
searchers  should  at  least  utilize  simulated  task  environments  similar  to  those  in  the 
operational  Navy.  This  will  require  observation  of  and  familiarization  with  teams 
in  operational  settings  (see  Sections  2.2  and  4.2). 
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Conclusion  5:  The  more  promising  lines  of  research  will  require  multivariate 
studies  of  performance  of  teams  over  time.  Such  studies  provide  an  integrated 
approach  to  team  research  not  adequately  achieved  by  more  narrow  studies  of 
individual  dimensions  of  team  performance.  The  research  will  include  development 
of  methods  for  simulating  operational  task  performance  in  the  laboratory,  con¬ 
trolled  experimental  studies  of  team  performance,  and  studies  of  performance 
evolution  (see  Sections  2.2  and  2.5). 

Conclusion  6:  Several  approaches  to  improving  team  performance  appear  to 
be  promising  areas  for  research  (see  Section  5).  These  include: 

1.  Improving  understanding  of  the  task  by  analyzing  performance  require¬ 
ments.  In  particular,  research  should  investigate  the  criteria  for  assessing 
performance,  the  knowledge  required  for  skilled  performance,  the  vari¬ 
ables  that  influence  or  limit  task  performance,  and  the  relationship  be¬ 
tween  individual  and  team  performance. 

2.  Improving  selection  of  team  members  by  matching  individual  skills  and 
abilities  to  task  demands. 

3.  Improving  training  and  exercising  of  both  individuals  and  teams  by  deve¬ 
loping  standard  training  methodologies,  feedback  procedures,  personal¬ 
ized  instruction  by  computer,  and  training  games. 

4.  Compensating  for  disruptive  effects  of  turnover  by  improving  training 
efficiency,  improving  socialization  and  integration  of  a  new  member  into 
the  team,  and  reducing  attrition. 

5.  Improving  the  organization  of  the  team  by  altering  standard  operating 
procedures,  communications  networks  and  protocols,  or  decisionmaking 
processes. 

6.  Developing  improved  human-machine  systems  and  aids  for  task  perfor¬ 
mance. 

Conclusion  7:  This  multivariate,  multidisciplinary  research  on  teams  re¬ 
quires  one  or  more  Team  Performance  Research  Centers  (see  Sections  2.5, 5.3,  and 
6.2).  These  Centers  must  have: 

1.  A  laboratory  facility  for  experimental  studies  of  team  performance. 

2.  A  system  for  representing  and  controlling  combat-like  situations  in  which 
teams  perform  their  tasks.  This  system  must  include  models  of  the  envi¬ 
ronment,  models  of  the  effects  of  the  team’s  and  enemy  actions,  mecha¬ 
nisms  for  dynamic  scenario  control,  an  interface  between  the  team  and  the 
environmental  model,  and  communications  links  among  team  members. 

3.  Advanced  computer  hardware  and  software  sufficient  to  simulate  impor¬ 
tant  functional  characteristics  of  operational  equipment  and  communica¬ 
tions. 

Conclusion  8:  Designing  a  Team  Research  Center  and  conducting  the  re¬ 
search  will  require  an  interdisciplinary  team  of  scientists.  This  team  must  include 
experts  in  cognitive  psychology,  social  psychology,  human  factors,  software  and 
systems  design,  artificial  intelligence,  simulation,  gaming,  and  instruction  (see  Sec¬ 
tion  6.1). 

Conclusion  9:  Basic  research  supported  by  the  program  should  be  comple¬ 
mented  by  policy  studies  to  determine  the  potential  impact  and  applicability  of 


research  results  in  the  operating  environment  (see  Sections  2.5, 4.1,  and  5.1).  These 
studies  should  include: 

1.  An  analysis  of  future  Navy  requirements  and  personnel  resources  to  de¬ 
termine  (a)  which  teams  are  most  critical  to  operational  effectiveness  and 
which  can  realize  significant  performance  improvements  from  research, 
and  (b)  how  procedures  and  training  will  need  to  be  adapted  to  the  abilities 
of  future  personnel. 

2.  An  analysis  of  the  barriers  to  translating  research  results  into  changes  in 
policy  and  procedures  in  operational  environments. 

Conclusion  10:  To  increase  the  impact  and  effectiveness  of  the  research  pro¬ 
gram,  ONR  should  include  the  targeted  user  and  applied  research  communities  in 
the  program  (see  Sections  4.3  and  4.5),  Representatives  of  these  communities  might 
serve  several  functions,  including: 

1.  Assisting  in  the  identification  of  critical  teams  for  study  and  the  determi¬ 
nation  of  highly  skilled  examples  of  such  teams. 

2.  Suggesting  problems  or  bottlenecks  in  improving  team  performance. 

3.  Facilitating  the  transition  of  research  results  into  applied  development 
efforts. 

4.  Providing  researchers  with  access  to  facilities  and  resources  in  operation¬ 
al  or  training  environments. 

Conclusion  11:  Implementation  of  the  proposed  program  should  proceed  in 
stages  (see  Section  6.3).  These  stages  include: 

1.  Immediate  initiation  of  a  study  to  select  the  task  domain(s)  for  inclusion 
in  the  program  and  to  specify  the  important  task  characteristics. 

2.  Identification  of  research  institutions  willing  and  able  to  undertake  a 
long-term  laboratory  development  and  interdisciplinary  research  effort. 

3.  Initiation  of  activities  to  coordinate  researchers  and  evaluate  research 
results,  including 

a. '  Formation  of  a  steering  committee  to  monitor  the  research  program. 

b.  Sponsorship  of  a  workshop  to  familiarize  contractors  with  the  pro¬ 
gram  structure,  task  domain,  performance  specifications,  and  oper¬ 
ational  equipment. 

c.  Insuring  ARPANET  access  for  all  contractors. 

4.  Initial  research  focused  on  the  development  of  laboratory  facilities,  simu¬ 
lations,  and  games. 

5.  A  program  management  decision,  approximately  three  years  into  the 
program,  about  how  to  continue.  The  decision  should  be  based  on  contrac¬ 
tors’  progress  in  constructing  laboratory  facilities,  the  quality  of  the  task 
representations,  initial  results  of  studies  using  the  laboratories,  and 
proposals  for  future  research. 

6.  Continuation  of  major  research  efforts  for  two  to  seven  years. 
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AIC  Air  Intercept  Controller 
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CAI  Computer-Assisted  Instruction 
CATCC  Carrier  Air  Traffic  Control  Center 
CATTS  Combined  Arms  Tactical  Training  Simulator 
CIC  Combat  Information  Center 
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EW  Electronic  Warfare 
FCSC  Fire  Control  Systems  Coordinator 
FPCC  Flag  Plot  Command  Center 
ICAI  Intelligent  Computer-Assisted  Instruction 
I/O  Input/Output 

ISD  Instructional  System  Development 
NAMMOS  Navy  Manpower  Mobilization  System 
NTDS  Naval  Tactical  Data  System 
NTEC  Naval  Training  Equipment  Center 
NTIS  National  Technical  Information  Service 
ONR  Office  of  Naval  Research 
R&D  Research  and  Development 
SAGE  Semi-Automatic  Ground  Environment 
SOP  Standard  Operating  Procedure 
SWC  Ship’s  Weapons  Coordinator 
TAO  Tactical  Action  Officer 
UBP  Underwater  Battery  Plot 
WES  Warfare  Environmental  Simulator 


1.  INTRODUCTION 


Many  Navy  operations  depend  upon  the  coordinated  actions  of  a  team  of  indi¬ 
viduals.  As  a  result,  the  quality  of  the  selection,  training,  equipment,  and  perfor¬ 
mance  of  teams  significantly  influences  the  effectiveness  of  Navy  operations.  Cur¬ 
rently,  several  obstacles  hinder  the  formation,  maintenance,  and  training  of  high- 
quality  Navy  teams.  These  obstacles  include: 

1.  Restrictions  on  available  training  time, 

2.  Rapidly  increasing  training  costs, 

3.  The  absence  of  data  on  the  cost-effectiveness  of  various  current  or  poten¬ 
tial  training  approaches, 

4.  The  lack  of  consistent  team  training  methods, 

5.  The  frequent  lack  of  specific  training  objectives  and  performance  criteria, 

6.  Declining  basic  skills  of  entering  Navy  personnel, 

7.  Increasing  complexity  of  equipment  and  jobs, 

8.  High  rates  of  personnel  turnover  on  teams  as  a  result  of  rotation  and 
attrition,  and 

9.  Organizational  disincentives  to  incorporating  new  training  technologies 
in  the  operational  environment. 

Against  this  background,  ONR  charged  the  Rand  Team  Performance  Project 
with  the  task  of  designing  of  a  research  program  aimed  at  the  improvement  of 
Navy  team  effectiveness.  ONR  requested  that  this  design  address  four  specific 
issues: 

1.  Research  needs  in  the  areas  of  team  training,  development,  and  evalu¬ 
ation. 

2.  Strategies  for  assuring  the  appropriate  mix  of  training,  organizational, 
and  human-factors  considerations  in  the  research  program. 

3.  The  roles  of  different  research  methodologies  in  the  program,  including 
theoretical  development,  modeling,  laboratory  experimentation,  field 
studies,  and  simulations. 

4.  The  utility  and  characteristics  of  a  dedicated  facility  for  research  on 
teams. 

To  determine  where  research  could  most  benefit  team  effectiveness,  we  first 
attempted  to  understand  the  obstacles  to  effectiveness  faced  by  the  Navy.  Among 
the  most  serious  of  the  Navy’s  problems  is  the  difficulty  of  attracting  and  retaining 
people  of  the  caliber  needed  to  operate  and  maintain  the  sophisticated  equipment 
vital  to  the  Navy  mission.  The  Office  of  the  Assistant  Secretary  of  Defense  for 
Manpower,  Reserve  Affairs,  and  Logistics  estimates  that  by  FY 1990  the  Navy  will 
acquire  between  7  and  24  percent  fewer  higher-quality1  males  than  they  obtained 
in  FY  1978  (OASD,  MRA&L,  1979).  The  lower  percentage  is  predicated  on  a  reces¬ 
sion  civilian  economy;  the  higher,  on  a  vigorous  civilian  economy.  At  the  same  time, 
technologies  and  equipment  used  in  military  operations  are  becoming  increasingly 

1  Defined  to  be  individuals  who  have  obtained  a  high-school  diploma  and  are  in  mental  groups  Mil. 
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complex  and  sophisticated.  Therefore,  a  major  focus  of  the  proposed  team  research 
program  is  to  determine  how  manipulations  in  team  structure,  redefinition  of  team 
procedures,  advances  in  training,  and  the  development  of  human-machine  perfor¬ 
mance  aids  can  best  utilize  the  abilities  of  future  Navy  personnel  and  equipment. 

Another  problem  facing  the  Navy  is  the  high  rate  of  personnel  turnover  on 
teams.  Much  of  this  turnover  is  caused  by  rotation  of  personnel  assignments.  In 
addition,  however,  many  recruits  are  lost  during  the  first  term  of  service.  OASD 
has  predicted  that  during  FY  1979  the  Navy  will  have  experienced  a  loss  of  28 
percent  of  active-duty  male  enlistees  before  the  end  of  their  first  three  years  of 
service.  Although  many  of  these  losses  occur  among  lower-quality  recruits,  such 
losses  place  heavy  burdens  on  Navy  training  and  operating  resources. 

People  familiar  with  the  Navy  often  attribute  these  personnel  problems  to  the 
discomforts  of  shipboard  life  and  the  isolation  of  persons  aboard  ship  from  the  rest 
of  society.  Yet  the  acquisition  and  first-term  attrition  rates  predicted  for  the  Navy 
are  not  appreciably  different  from  the  corresponding  figures  for  the  Army,  which 
does  not  require  sea  duty.  Nevertheless,  we  believe  that  work  environment  signifi¬ 
cantly  affects  morale  and  job  satisfaction.  Therefore,  another  focus  for  the  team 
research  program  is  the  determination  of  whether  and  how  the  formation,  training, 
and  exercising  of  teams  can  improve  morale  and  job  satisfaction  aboard  ship. 

A  third  problem  facing  the  Navy  is  the  current  lack  of  adequate  training  time 
and  resources.  The  declining  quality  of  entering  recruits  has  increased  the  demand 
for  training  in  basic  skills.  This,  among  other  factors,  has  reduced  the  time  available 
for  job-related  training,  particularly  team  training.  Further,  few  facilities  exist  for 
team  training,  and  the  equipment  in  those  facilities  that  do  exist  is  often  not  similar 
to  shipboard  equipment.  Finally,  training  curricula  often  lack  systematic  ap¬ 
proaches  to  instruction,  feedba'ck,  and  performance  evaluation.  Thus,  another  focus 
of  the  proposed  research  program  is  the  enrichment  of  training  for  teams. 

It  is  clear  that  the  Navy  will  need  to  adopt  a  variety  of  measures  to  alleviate 
these  problems.  Some  of  these  measures,  such  as  revised  recruiting,  training,  and 
rotation  policies,  may  require  policy  changes  in  overall  Navy  personnel  manage¬ 
ment.  Others  will  require  the  discovery  of  new  knowledge  and  the  development  of 
technology  based  on  research.  However,  the  ultimate  goal  of  improving  team  effec¬ 
tiveness  is  obviously  too  difficult  and  general  to  be  achieved  solely  through  basic 
research  efforts.  Producing  a  measurable  improvement  in  Navy  team  performance 
requires  the  selection  of  research  problems  of  concern  to  the  operating  Navy,  the 
orchestration  of  a  set  of  coordinated  activities  in  all  6.1-6.5  research  and  develop¬ 
ment  categories  (research,  exploratory  development,  advanced  development,  engi¬ 
neering  development,  management  and  support),  and  the  acceptance  and  im¬ 
plementation  of  R&D  products  by  operational  personnel.  Consequently,  we  broad¬ 
ened  our  task  to  include  the  design  of  a  program  in  which  research  results  will  lead 
directly  to  changes  in  equipment  or  procedures  in  operational  environments. 

Such  technology  transfer  will  depend  both  on  the  success  of  the  research  and 
on  the  ability  of  ONR  to  move  results  down  the  R&D  pipeline.  For  the  research 
program  to  succeed,  it  must  incorporate  methods  for  improving  the  impact  of 
research  results.  Thus,  this  document  outlines  a  broad  organizational  plan  for  the 
management  of  the  research  program,  in  addition  to  a  set  of  recommended  specific 
topics  for  research. 

In  attempting  to  design  a  program  structure  and  select  a  set  of  research  topics, 
we  considered  a  variety  of  program  objectives  that  would  enhance  the  prospects 
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for  success  of  the  research  program.  We  ultimately  selected  six  objectives — three 
to  be  met  by  the  overall  design  of  the  program  and  three  to  be  met  by  the  successful 
execution  of  the  program  research: 

Program  objectives 

1.  To  limit  the  scope  of  the  research  program  to  the  investigation  of  a  small 
set  of  critical  Navy  teams  and  research  problems. 

2.  To  support  research  that  would  attract  the  highest-quality  scientists  and 
that  would  have  early  applicability  to  operational  needs. 

3.  To  encourage  and  facilitate  the  feed-forward  of  promising  research  results 
to  subsequent  research  efforts  (within  ONR)  and  applied  development  and 
technology  transfer  efforts  (in  6.2-6.5  programs). 

Research  objectives 

4.  To  develop  analytical  and  theoretical  models  of  team  performance  (that 
is,  to  answer  the  question,  What  does  the  team  do?) 

5.  To  improve  the  evaluation  of  team  performance  (that  is,  to  answer  the 
question,  How  well  is  the  team  performing?) 

6.  To  improve  team  performance  (that  is,  to  answer  the  question,  How  might 
the  team  perform  better?) 

These  considerations  motivated  the  conclusions  about  program  structure,  man¬ 
agement  policies,  and  research  topics  discussed  in  the  following  sections.  Section 
2  provides  an  overview  of  the  proposed  research  program  and  discusses  the  aspects 
of  current  Navy  teams  and  team  activities  that  motivated  the  conclusions  and 
recommendations  of  this  report.  Section  3  presents  the  project  conclusions  and 
general  recommendations  for  the  design,  management,  and  content  of  the  research 
program.  Section  4  discusses  in  detail  specific  recommendations  for  ONR  manage¬ 
ment  activities  to  implement  the  program.  Section  5  presents  the  set  of  specific 
research  areas  that  seem  most  promising  and  that  should  be  supported  within  the 
program.  Finally,  Section  6  outlines  a  time-phased  plan,  budgetary  guidelines,  and 
resource  requirements  for  the  program. 


2.  AN  OVERVIEW  OF  THE  PROGRAM  DESIGN 


2.1.  THE  TEAM  CONCEPT 

To  design  a  research  program  on  teams,  we  were  forced  to  consider  at  the  outset 
how  to  characterize  a  "team.”  Since  recent  reviews  of  military  teams  have  dis¬ 
cussed  the  concept  of  a  team  in  some  detail  (Hall  and  Rizzo,  1975;  Wagner,  Hibbits, 
Rosenblatt,  and  Schulz,  1976),  we  elected  not  to  seek  a  precise  definition  or  to  make 
fine  distinctions  between  teams  and  other  multi-individual  groups  (e.g.,  crews, 
groups,  units).  Rather,  we  use  the  term  "team”  generically  to  refer  to  a  set  of 
individuals  working  cooperatively  to  achieve  some  common  objective.  We  have 
limited  the  scope  of  our  consideration  to  teams  with  a  sufficiently  small  member¬ 
ship  that,  in  principle,  all  members  of  the  team  could  be  exercised  simultaneously. 

This  concept  of  a  team  embraces  such  diverse  Navy  functional  groups  as  air 
crews,  gunnery  teams,  air  traffic  control  teams,  maintenance  units,  mess  crews, 
combat  information  center  (CIC)  teams,  underwater  battery  plot  teams  (submarine 
tracking  and  attacking),  ship  maneuvering  teams,  and  propulsion  and  powerplant 
teams.  These  teams  may  vary  in  any  of  several  critical  attributes  characterizing 
team  composition,  function,  and  organization.  Over  the  years,  researchers  have 
developed  taxonomies  to  capture  the  important  dimensions  of  variation  (Glanzer 
and  Glaser,  1955;  Haggard,  1963;  Engel,  1970;  Meister,  1976).  However,  Meister 
(1976)  concludes  that  "a  satisfactory  taxonomy  of  team  dimensions  has  not  yet  been 
developed”.  While  we  did  not  attempt  to  develop  a  definitive  taxonomy  of  teams, 
we  did  seek  to  identify  some  of  the  attributes  that  could  describe  teams  with 
different  function&dnd  structures.  These  attributes  could  serve  as  a  basis  for  loosely 
clustering  teams  with  similar  performance  characteristics  and  research  require¬ 
ments.  The  following  attributes  seemed  to  capture  much  of  the  variation  among 
teams: 

1.  Team  Size.  Variations  in  team  size  influence  both  the  complexity  in  team 
communications  and  the  feasibility  of  training  teams  in  toto. 

2.  Team  Function.  Teams  vary  in  the  nature  of  their  principal  activities. 
These  activities  include  performing  perceptual  discriminations  (e.g.,  a 
radar  team),  utilizing  perceptual-motor  skills  (e.g.,  a  gun-loading  team), 
and  processing  symbolic  information  and  making  decisions  (e.g.,  a  CIC 
team). 

3.  Performance  Sequencing.  The  synchrony  of  team  member  performance 
influences  the  requirements  for  control  or  supervision  of  behaviors.  Team 
members  may  perform  simultaneously,  sequentially,  or  asynchronously. 

4.  Performance  Integration.  Teams  vary  in  the  extent  to  which  members 
cooperate  during  performance.  Tasks  may  require  considerable  interac¬ 
tion,  little  interaction  but  the  sharing  of  resources,  or  relatively  indepen¬ 
dent  behaviors.  These  requirements  influence  the  extent  to  which  overall 
team  performance  requires  coordination  skills  and  the  extent  to  which 
overall  performance  is  predictable  from  aggregate  J  individual  perfor- 


5.  Command/Communications  Structure.  Teams  vary  in  their  command 
and  communications  regimes,  particularly  in  the  extent  to  which  specific 
individuals  act  as  central  points  for  receiving  messages  and  making  deci¬ 
sions.  Alternative  structures  include  hierarchical,  linear,  star,  or  modula¬ 
rized  organizations. 

6.  Role  Flexibility.  Teams  vary  in  the  extent  to  which  the  behaviors  of  the 
different  team  members  are  rigidly  fixed  and  segregated.  The  flexibility  of 
team  roles  will  influence  the  extent  to  which  team  tasks  can  potentially 
be  reorganized  to  improve  efficiency  or  performance. 

Since  teams  may  vary  so  widely  in  structure  and  function,  the  requirements  for 
selecting  team  members,  organizing  and  training  the  team,  and  measuring  team 
performance  will  vary  as  a  function  of  team  characteristics.  Thus,  no  research 
program  can  address  the  needs  of  all  Navy  teams. 


2.2.  SELECTION  OF  TARGET  TEAMS 

We  were  next  faced  with  the  decision  of  whether  to  design  a  program  based  on 
a  broad-spectrum  approach  to  teams  or  on  a  coordinated,  in-depth  study  of  a  select¬ 
ed  team  or  team  type.  The  broad-spectrum  approach  offers  the  advantage  of 
greater  flexibility  for  contractors  to  select  their  own  research  focus.  In  addition,  this 
approach  would  provide  better  research  coverage  of  the  variety  of  team  types 
within  the  Navy.  On  balance,  however,  the  selection  of  a  more  narrow  research 
domain  seemed  more  consistent  with  ONR’s  goals  and  seemed  to  offer  a  better 
chance  of  producing  usable  results.  Several  considerations  supported  this  conclu¬ 
sion: 

1.  A  program  with  a  major,  focused  thrust  in  a  single  domain  would  have 
higher  visibility  and,  if  successful,  higher  impact  than  a  less  focused  effort. 

2.  A  single  research  focus  would  offer  the  opportunity  for  significant  coopera¬ 
tion  and  synergy  among  contractors. 

3.  A  single  research  focus  would  not  limit  the  applicability  of  successful 
research  results.  Rather,  successful  theories,  methods,  and  technologies 
developed  in  the  chosen  domain  could  be  transferred  to  new  domains.* 

The  selection  of  a  particular  focus  for  the  research  program  required  both  a 
review  of  prior  research  on  teams  and  a  consideration  of  the  types  of  actual  Navy 
teams.  We  conducted  an  extensive  search  of  the  literature  and  compiled  a  library 
of  documents  on  teams.  Sources  of  references  included  Psychological  Abstracts, 
Sociological  Abstracts,  National  Technical  Information  Service  (NTIS),  Defense 
Documentation  Center  (DDC),  Educational  Resources  Information  Center  (ERIC), 
Social  Sciences  Citation  Index,  the  Rand  library,  and  various  documents  recom¬ 
mended  by  the  Team  Performance  Project  staff.  We  did  not  attempt  a  comprehen¬ 
sive,  formal  review  of  this  literature,  since  several  recent  reviews  have  provided 
excellent  digests  of  this  work  (Alexander  and  Copperband,  1965;  Hall  and  Rizzo, 

1  This  strategy  proved  successful  in  the  Defense  Advanced  Research  Projects  Agency  (ARPA)  speech 
understanding  program.  Moreover,  the  system  architecture  developed  for  speech  understanding  in  the 
Carnegie-Mellon  HEARSAY-II  program  has  been  adapted  for  use  in  subsequent  efforts  to  create  image 
understanding  systems. 
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1975;  Meister,  1976;  Wagner,  Hibbits,  Rosenblat,  and  Shulz,  1976).  However,  we  did 
examine  the  domains,  methodologies,  and  conclusions  of  these  efforts  to  determine 
the  current  state  of  knowledge  about  teams.  (A  bibliography  of  these  sources  ap¬ 
pears  at  the  end  of  this  report.) 

The  Team  Performance  Project  also  conducted  a  three-day  workshop  on  teams 
at  Rand  on  November  27-29,  1980.  The  workshop  comprised  a  set  of  sixteen 
presented  papers  that  described  past,  current,  and  potential  future  research  on 
teams  in  a  variety  of  disciplines.  In  addition,  there  were  wide-ranging  discussions 
among  the  participants,  the  Rand  project  staff,  and  various  Department  of  Defense 
personnel. 

Several  conclusions  emerged  from  the  literature  and  the  workshop.  First,  an 
enormous  body  of  research  has  addressed  various  aspects  of  team  composition, 
structure,  training,  and  performance.  Second,  few  of  the  research  results  appear  to 
be  applicable  to  the  solution  of  real  Navy  team  problems.  In  general,  prior  research 
has  investigated  teams  dissimilar  in  structure,  purpose,  and  activities  to  Navy 
teams.  Such  research  typically  focuses  on  a  small  set  of  variables,  a  narrow  range 
of  task  and  environmental  variations,  and  simplistic  laboratory  methods.  Finally, 
the  tested  hypotheses  and  models  are  frequently  too  abstract  or  discipline-bound 
to  offer  prescriptions  for  teams  in  the  operational  environment. 

Consequently,  we  concluded  that  the  research  program  should  focus  on  the 
study  of  a  team  or  type  of  team  similar  to  one  that  actually  exists  in  the  Navy.  Since 
research  using  actual  operational  teams  and  equipment  may  be  problematic  for 
many  potential  contractors,  the  teams  actually  studied  may  only  simulate  critical 
functional  characteristics  of  the  operational  teams.  Nevertheless,  the  task  environ¬ 
ments  investigated  by  contractors  should  be  motivated  and  constrained  by  the 
properties  of  particular  Navy  teams. 

Three  criteria  guided  the  search  for  an  appropriate  domain  for  intensive  inves¬ 
tigation.  First,  the  program  area  had  to  be  one  in  which  successful  research  might 
lead  to  significant  improvements  in  the  performance  of  the  teams  under  investiga¬ 
tion.  This  suggested  that  attractive  candidates  would  be  teams  with  considerable 
variation  in  performance,  poorly  specified  performance  metrics  and  criteria,  and 
flexibility  in  operating  procedures  and  training  methods.  Such  teams  might  be  able 
and  willing  to  alter  their  activities  based  on  recommendations  from  research.  Fur¬ 
ther,  teams  that  perform  in  a  technologically  changing  environment  (e.g.,  comput¬ 
ers,  communications,  weapons)  face  changes  in  procedures  as  equipment  is  re¬ 
placed.  These  teams  offer  an  excellent  opportunity  to  introduce  innovations  in 
training,  task  allocation,  and  performance  aids. 

The  second  criterion  for  the  selected  domain  was  that  team  performance  must 
depend  on  more  than  simply  the  aggregated  performance  of  the  individuals.  Other¬ 
wise,  little  would  be  gained  in  team  research  beyond  what  might  be  accomplished 
in  the  improved  training  of  individuals.  This  criterion  suggested  the  desirability  of 
selecting  a  team  or  type  of  team  that  requires  extensive  interaction  among  team 
members  and  that  uses  poorly  specified  or  highly  flexible  operating  procedures. 

The  third  criterion  for  selecting  a  research  domain  was  that  it  be  an  area  in 
which  improvements  in  team  performance  would  significantly  improve  overall 
Navy  survivability  and  combat  effectiveness.  Thus,  research  should  focus  on  teams 
whose  performance  is  vital  to  the  success  of  the  Navy’s  missions. 

These  considerations  and  the  program  objectives  outlined  in  Section  1  led  us 
to  the  following  conclusion; 
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The  greatest  leverage  in  team  performance  research  can  be  attained  by 

focusing  research  on  teams  that  receive  and  evaluate  dynamic  information 

and  perform  time-stressed  decisionmaking. 

Such  teams  deal  with  the  environment  largely  through  symbolic  representation 
for  the  purpose  of  threat  detection  and  evaluation,  situation  assessment,  planning, 
decisionmaking,  and  control  (plan  execution  and  monitoring).  These  teams  tend  to 
have  the  following  attributes: 

1.  They  rely  primarily  on  information-processing  activities  rather  than  per¬ 
ceptual-motor  skills. 

2.  They  require  the  coordinated  activities  of  several  subteams,  such  as  those 
that  process  sensor  data,  display  the  data,  use  the  data  for  decisionmak¬ 
ing,  or  allocate  and  direct  team  resources. 

3.  They  require  a  high  degree  of  interaction  among  team  members  and 
subteams. 

4.  They  utilize  relatively  loosely  structured  standard  operating  procedures 
(SOPs)  because  their  data  are  highly  volatile  and  time-dependent. 

5.  They  are  flexible  because  they  must  respond  to  surprises,  emergencies, 
and  a  variety  of  external  situations. 

6.  Their  activity  level  varies  greatly,  from  routine  operations  to  critical 
decisionmaking  under  conditions  of  heavy  load. 

The  next  section  describes  some  shipboard  Navy  teams  that  perform  this  type 
of  decisionmaking. 


2.3.  INFORMATION-PROCESSING  AND  DECISIONMAKING 
TEAMS  IN  THE  NAVY 

Each  Navy  ship  comprises  several  closely  integrated  teams  that  contribute  to 
command  decisionmaking.  While  the  details  of  team  organization,  procedures,  and 
equipment  vary  from  ship  to  ship,  the  general  descriptions  presented  here  illus¬ 
trate  the  variety  of  tasks  and  functions  that  these  teams  perform. 

The  CIC  is  the  focus  of  each  ship’s  tactical  information  processing  and  decision¬ 
making.  CIC  teams  provide  command  and  control  functions  for  the  ship  and  for 
shipboard  and  other  aircraft  in  the  immediate  area.  They  detect  and  track  surface 
ships,  submarines,  and  aircraft;  they  direct  ship  maneuvers;  they  monitor  the 
environment  and  the  progress  of  the  battle;  they  control  the  ship’s  weapons  deploy¬ 
ment;  and  they  communicate  with  other  CICs  to  coordinate  battle  group  activities.2 

While  the  commanding  officer  of  the  ship  has  ultimate  command  authority, 
much  of  the  tactical  data  evaluation  and  decisionmaking  in  the  CIC  is  performed 
by  the  Tactical  Action  Officer  (TAO).  He  must  evaluate  threats,  allocate  resources 
to  meet  the  threats,  and  make  ship  maneuvering  decisions.  Three  data  sources 
provide  inputs  to  his  decisionmaking  process:  air  radar  information,  surface  and 

*  Navy  documents  that  describe  these  tactical  command  and  control  procedures  include  Combat 
Information  Center  Doctrine  (prepared  by  individual  ships),  Combat  System  Operation  Design  (pre¬ 
pared  by  Commander,  Naval  Sea  Systems  Command,  for  some  classes  of  ships),  and  Staff  Manuals  (such 
as  that  prepared  by  Fleet  Combat  Direction  Systems  Support  Activity,  Dam  Neck,  Virginia,  for  the 
Navy  Tactical  Data  System). 
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sub-surface  radar  and  sonar  information,  and  electronic  warfare  information. 
These  three  types  of  information  are  displayed  more  or  less  independently  and 
separately  in  the  CIC. 

The  ship’s  weapons  coordinator  (SWC)  coordinates  the  air  and  surface  war 
under  the  direction  of  the  TAO.  He  views  a  computer-generated  symbolic  display 
of  targets  and  their  movements,  and  he  assigns  responsibility  for  some  air  targets 
to  the  air  intercept  controllers  (AICs).  The  AICs  communicate  targets,  commands, 
and  locations,  etc.,  to  the  combat  air  patrol  under  the  ship’s  control.  The  SWC 
assigns  responsibility  for  the  tracking  and  engagement  of  other  air  and  surface 
targets  to  the  fire  control  systems  coordinator  (FCSC)  in  the  weapons  area.  The 
FCSC  controls  the  shipboard  missile  and  gun  systems  and  radars  and  supervises 
the  activities  of  the  operators  of  these  systems. 

On  aircraft  carriers,  the  movement  of  aircraft  on  and  in  the  vicinity  of  the 
carrier  is  managed  by  the  carrier  air  traffic  control  center  (CATCC).  This  team, 
which  is  not  located  in  the  CIC,  provides  air  approach  control  and  air  launch  and 
recovery  direction  for  carrier  aircraft.  Under  the  direction  of  the  air  squadron 
leader,  the  CATCC  must  coordinate  its  activities  with  the  CIC  and  the  bridge. 

The  radar  detection  and  tracking  area  maintains  the  updated  radar  displays  for 
the  CIC.  Search  radar  operators  view  radar  repeaters  and  detect  the  presence  of 
objects  (ships,  aircraft,  missiles).  Track  identifiers  attempt  to  determine  the  identity 
of  these  objects  and  whether  they  are  friendly  or  hostile  "platforms.”  The  track 
supervisor  coordinates  these  activities  and  communicates  with  the  SWC  and  other 
ships  to  determine  the  current  status  of  these  platforms. 

The  electronic  warfare  (EW)  area  comprises  several  individuals  who  detect 
electronic  emissions  from  remote  air  targets.  These  emissions  indicate  the  bearing 
and  identity  of  the  targets.  This  information  is  communicated  to  a  team  member 
in  the  CIC  who  maintains  current  EW  information  on  a  Plexiglas  status  board.  The 
TAO  and  SWC  integrate  this  information  with  radar  displays  to  pinpoint  the  loca¬ 
tion  and  identity  of  enemy  targets. 

Surface  and  subsurface  information  is  aggregated  and  evaluated  in  the  surface 
area  of  the  CIC.  The  surface  watch  officer  coordinates  the  surface  war  and  anti¬ 
submarine  war  (ASW)  under  the  direction  of  the  TAO.  The  surface  area  maintains 
a  horizontal,  manually  operated  plotting  board  to  display  the  locations  and  move¬ 
ments  of  surface  ships  and  submarines.  Information  for  maintaining  this  display 
is  provided  by  radar  and  by  the  sonar  area,  located  outside  of  the  CIC.  The  sonar 
area,  like  the  radar  area,  comprises  several  individuals  who  receive  low-level  sen¬ 
sor  data  and  attempt  Co  determine  target  identities  and  locations.  Requests  for 
weapons  engagements  are  sent  to  the  anti-submarine  air  controller  and  to  the 
underwater  battery  plot  (UBP),  also  located  outside  the  CIC. 

Each  ship  performs  this  variety  of  CIC  functions.  The  flag  plot  command  center 
(FPCC)  is  a  CIC,  commanded  by  the  battle  group  admiral,  that  orchestrates  the 
activities  of  the  entire  battle  group.  The  FPCC  is  typically  located  on  the  aircraft 
carrier  for  the  battle  group.  It  formulates  the  fleet  battle  plan  and  monitors  its 
execution  through  communications  with  the  fleet  EW  coordinator,  the  fleet  ASW 
coordinator,  the  fleet  surface  warfare  coordinator,  and  the  fleet  air  warfare  coor¬ 
dinator.  These  fleet  coordinators,  in  turn,  must  coordinate  the  activities  of  the 
various  CICs  in  the  battle  group. 
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2.4.  THE  DEMAND  FOR  RESEARCH  ON  DECISIONMAKING  TEAMS 

Each  of  the  Services  has  teams  that  perform  similar  command  decisionmaking 
functions.  Such  teams  are  particularly  important  to  military  operations  because 
they  make  decisions  and  initiate  actions  that  affect  the  positioning,  the  weapons 
deployment,  the  combat  readiness,  and  the  ultimate  survivability  of  entire  oper¬ 
ational  units.  The  criticality  of  these  teams  is  reflected  in  the  heavy  investment  in 
studying  and  training  them,  both  in  the  research  community  and  in  operational 
environments. 

For  example,  a  long  history  of  man-machine  studies  have  investigated  the 
training  of  such  teams  (Parsons,  1972),  focusing  on  different  aspects  of  team  perfor¬ 
mance,  including  operator  capacity  to  process  information,  evaluations  of  informa¬ 
tion  displays,  decisionmaking  processes,  communications  procedures,  and  evalua¬ 
tions  of  computer  support  options.  Such  research  has  primarily  investigated  indi¬ 
vidual  performance  in  team  settings  rather  than  team  training  or  performance  per 
se.  Project  Cadillac  (New  York  University)  and  The  Rand  Corporation’s  Systems 
Research  Laboratory,  among  others,  pioneered  broad-spectrum  approaches  to 
team  training  and  performance.  Many  other  efforts,  such  as  the  Semi-Automatic 
Ground  Environment  (SAGE)  system  (Systems  Development  Corporation),  focused 
more  narrowly  on  specific  training  issues,  methodologies,  or  hardware/software 
configurations.  In  general,  these  efforts  have  shown  that  useful  research  results  are 
most  likely  to  derive  from  a  systems  approach  to  multiple  aspects  of  performance 
rather  than  from  studies  confined  to  a  few  team  attributes.  This  suggests  that  to 
be  successful,  new  research  on  teams  must  emphasize  broad,  multivariate,  interdis¬ 
ciplinary  studies  of  performance. 

More  recently,  numerous  facilities  have  been  developed  and  studies  undertak¬ 
en  to  investigate  complex  team  decisionmaking,  including  the  Battle  Management 
Laboratory  developed  at  Rand  for  the  Air  Force,  studies  of  Air  Force  battle  man¬ 
agement  decisionmaking  at  MITRE  (Wohl,  1979),  the  formation  of  an  Air  Force 
Battle  Simulation  Working  Group  to  improve  simulation  capabilities  (AFBSWG, 
1979),  the  development  of  various  tactical  decision  aids  for  the  Navy  (reviewed  in 
Miller,  Rice,  and  Metcalf,  1979),  and  the  development  and  evaluation  of  various 
Army  team  training  systems  (Thurmond  and  Kribs,  1978;  Scott,  Meliza,  Hardy, 
Banks,  and  Word,  1979;  Meliza,  Scott,  and  Epstein,  1979).  In  addition,  the  Navy 
maintains  several  facilities  for  team  training  in  complex  shipboard  decisionmaking 
(Rizzo,  1980). 

The  importance  of  research  on  teams  with  command  and  control  functions  will 
continue  to  grow.  Sophisticated  information-gathering  systems  will  extend  the 
amount,  scope,  and  type  of  information  available  from  sensors.  This  will  extend  and 
change  the  demands  on  individuals  to  integrate  various  data  and  assess  the  current 
situation.  More  sophisticated  weapons  systems  will  require  increased  speed  and 
accuracy  of  deployment  decisions.  New  hardware  and  software  capabilities  will 
become  available  to  support  communications  among  team  members,  cooperative 
problem  solving  and  decisionmaking,  and  interfaces  among  sensors,  weapons,  and 
humans.  Finally,  the  increasing  complexity  of  systems  and  procedures  will  make 
it  increasingly  difficult  to  analyze,  assess,  and  prescribe  methods  for  performance. 
These  trends  lead  to  a  growing  requirement  for  research  aimed  at  achieving  highly 
effective  performance  by  such  teams.  Consequently,  we  recommend  the  study  of 
these  teams  in  the  team  performance  program. 
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2.5.  THE  FRAMEWORK  FOR  TEAM  PERFORMANCE  RESEARCH 

As  discussed  in  Section  2.2,  the  principal  shortcomings  of  prior  team  research 
are  (1)  the  lack  of  similarity  between  laboratory  teams  and  operational  teams  and 
(2)  the  narrow  focus  and  methodology  of  most  laboratory  studies.  We  sought  a 
framework  and  design  for  the  research  program  that  would  avoid  these  shortcom¬ 
ings;  after  considering  several  designs,  we  arrived  at  one  that  we  believed  could 
best  meet  the  program  objectives  outlined  in  Section  1.  Figure  1  summarizes  this 
framework. 

The  center  of  Fig.  1  represents  the  research  and  development  pyramid  that 
generates  new  systems,  ideas,  and  procedures  for  the  military  Services.  The  vari¬ 
ous  levels  in  the  pyramid  are  labeled  on  the  left  side  of  the  figure  with  their  R&D 
category  designations.  At  the  top  of  the  pyramid  are  the  operational  teams  targeted 
for  improvement  by  the  research  program.  (CIC  and  CATCC  teams  are  merely 
illustrative  of  the  information-processing,  decisionmaking  teams  we  recommend 
above  as  potential  research  targets.)  The  next  three  levels  in  the  pyramid  represent 
the  various  stages  in  advancement  of  the  development  of  systems  and  ideas  for  use 
in  operational  environments.  The  bottom  of  the  pyramid  represents  6.1  research, 
the  type  of  scientific  studies  typically  sponsored  by  ONR. 

In  this  framework,  the  selection  of  target  teams  would  constrain  the  set  of 
potential  6.1  research  topics  (indicated  by  the  ovals).  As  in  any  research  program, 
only  some  of  the  research  at  the  bottom  of  the  pyramid  will  produce  successful 
results  and  be  carried  forward  into  6.2  development  studies.  Such  selective  filtering 
would  occur  at  each  stage  in  the  pyramid.  As  results  move  up  the  pyramid,  the 
objectives  of  the  work  become  more  applied  and  more  specific  to  particular  appli¬ 
cations.  In  addition,  however,  successful  and  promising  research  results  at  the 
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THE  R&D  PYRAMID 


PROPOSED  TEAM 
PERFORMANCE  STUDIES 


Fig.  1— A  framework  for  team  performance  research 
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bottom  of  the  pyramid  will  presumably  suggest  new  applications  both  in  the  Navy 
and  in  the  other  Services.  Thus,  research  will  generalize  to  domains  and  appli¬ 
cations  beyond  those  targeted  by  the  research  program. 

We  believe  that  successful  research  on  command  decisionmaking  teams  will 
require  the  coordination  of  a  variety  of  research  approaches  in  a  simulated  task 
environment.  As  previous  studies  of  man-machine  systems  illustrate,  the  establish¬ 
ment  of  facilities  and  capabilities  required  to  simulate  a  complex  task  environment 
is  itself  a  substantial  undertaking  (Parsons,  1972).  Therefore,  we  concluded  that 
research  projects  conducted  in  the  program  should  be  clustered  in  large-budget 
research  centers.  The  actual  number  of  such  centers  that  ONR  could  support  would 
depend  on  the  budget  available  for  the  program  and  the  proposed  size  of  each 
center.  (Section  6  discusses  the  anticipated  requirements  of  these  centers).  Each 
center  would  create  a  task  simulation  environment  and  conduct  a  number  of  re¬ 
lated  but  distinct  interdisciplinary  studies  of  team  performance.  As  shown  in  Fig. 
1,  the  activities  of  these  centers  would  overlap  to  the  extent  that  different  centers 
develop  laboratories  with  similar  equipment  or  for  the  study  of  the  same  teams. 
However,  each  site  would  presumably  investigate  a  unique  set  of  research  ques¬ 
tions. 

We  believe  that  three  classes  of  studies  in  the  research  program  could  lead  to 
significant  improvements  in  team  effectiveness.  These  classes  differ  in  the  location 
in  the  R&D  pyramid  at  which  improvements  would  occur.  They  are  indicated  on 
the  right-hand  side  of  Fig.  1. 

Organizational  policy  studies  would  focus  on  current  operational  teams  and 
Navy  policies  and  plans  that  may  affect  their  performance  and  shape  the  basic 
research  of  the  next  five  to  ten  years.  Several  studies  of  Navy  operations,  discussed 
in  Section  5.1,  may  produce  recommendations  for  improving  team  effectiveness. 
These  include  assessing  the  requirements  and  characteristics  of  future  Navy  teams, 
understanding  the  organizational  factors  involved  in  introducing  innovation,  and 
assessing  the  causes  and  effects  of  personnel  turnover  on  teams. 

The  second  class  of  studies  would  focus  on  the  translation  of  current  knowledge 
and  technologies  into  the  operational  environment.  Section  5.1.3  presents  several 
examples  of  current  technologies  that  could  immediately  improve  the  performance 
of  Navy  teams.  This  class  of  studies  would  develop  plans  and  specific  methods  for 
introducing  these  improvements  into  operational  environments. 

Finally,  the  bulk  of  the  research  activities  proposed  in  Section  5  (5.2-5  8)  focus 
on  new  studies  of  team  performance.  These  studies  would  include  laboratory  design 
and  development,  theoretical  development,  simulations  and  games  to  investigate 
team  interactions  and  man-machine  communications,  experiments  in  manipulating 
variables  of  theoretical  interest,  and  research  on  a  variety  of  software  aids  for 
performance  modeling,  training,  and  improving  task  performance.  The  recom¬ 
mended  studies  are  based  on  evaluations  of  the  current  state  of  development  in 
psychology,  computer  science,  simulation,  and  military  gaming. 


2.6.  PRODUCTS  OF  THE  RESEARCH  PROGRAM 

The  ultimate  outcomes  of  the  research  program  will  be  measured  by  its  prod¬ 
ucts.  The  quality  of  these  products  will  be  influenced  by  many  as  yet  undetermined 
factors,  including  program  budget,  program  longevity,  contractors,  the  content  of 
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the  research,  and  the  success  of  the  research.  Despite  the  uncertainties,  we  antici¬ 
pate  that  the  research  program  should  support  four  types  of  products:  scientific 
knowledge,  operational  norms,  designs  for  technology  transfer,  and  laboratory 
facilities.  Each  of  these  products  and  their  consumers  are  discussed  below. 

The  scientists  participating  in  the  research  will  produce  and  document  results 
for  the  scientific  community  in  technical  reports,  journal  publications,  books,  and 
conference  presentations  and/ or  proceedings.  The  knowledge  and  theories  derived 
from  the  research  should  benefit  a  scientific  community  that  extends  beyond  indi¬ 
viduals  involved  in  the  study  of  teams,  particular  teams,  or  military  problems. 

Successful  research  results  should  also  be  aggregated  into  documents  for  use 
by  the  operational  community.  Thus,  contractors  might  produce  a  manual  for  Navy 
commanders  that  provides  a  practical  guide  on  how  to  train,  reward,  rotate,  evalu¬ 
ate,  or  organize  particular  shipboard  teams.  The  recommendations  in  such  a  docu¬ 
ment  would  be  based  on  behavioral  principles  derived  from  the  research. 

Some  research  results  might  take  the  form  of  promising  technologies  for  aiding 
teams,  including  hardware  configurations,  design  architectures,  man-machine  in¬ 
terfaces  based  on  human  factors  studies,  computer-assisted  training  methods  or 
programs,  and/or  demonstration  systems  for  supplementing  or  replacing  team 
functions.  Thus,  researchers  might  produce  hardware  or  software  specifications 
and  designs  for  technology  transfer  to  further  development  efforts. 

Finally,  the  research  program  will  support  the  development  of  laboratory  facili¬ 
ties  and  capabilities.  These  laboratories  might  be  used  in  subsequent  research  and 
development  by  the  same  or  other  contractors,  or  they  might  be  delivered  and 
installed  at  Navy  training  sites  for  use  in  operational  training  and  training  develop¬ 
ment. 


3.  GENERAL  CONCLUSIONS 


Guided  by  the  six  goals  presented  in  Section  1  and  the  considerations  discussed 
in  Section  2,  we  arrived  at  a  set  of  general  conclusions  about  a  program  structure 
offering  the  best  chance  of  success.  These  conclusions  constrained  and  suggested 
the  specific  recommendations  for  activities  to  be  undertaken  by  program  manage¬ 
ment  and  projects  to  be  sponsored  in  the  research  program.  We  give  here  our  set 
of  general  conclusions.  In  the  remainder  of  the  report  we  elaborate  these  conclu¬ 
sions  as  recommendations  for  specific  research  and  program  management  activi¬ 
ties. 

Conclusion  1:  The  teams  for  which  ONR-supported  research  is  likely  to  have 
the  greatest  impact  are  those  that  process  large  amounts  of  symbolic  information 
and  make  tactical  decisions  under  considerable  time-stress.  Such  teams  include 
those  in  the  tactical  flag  command  centers,  CICs,  nuclear  submarine  attack  centers, 
ASW  attack  centers,  and  carrier  and  shore-based  air  traffic  control  centers.  These 
teams  are  the  most  critical  for  combat  effectiveness  and  ship  survivability  and  are 
ubiquitous  in  the  Navy  and  in  the  other  Services.  Moreover,  their  effectiveness 
depends  on  the  coordination  and  organization  of  the  team  in  addition  to  the  skilled 
performance  of  individual  team  members. 

Conclusion  2:  Current  personnel  problems  and  Navy  policies  limit  team  per¬ 
formance.  Research  should  identify  organizational  policies,  technological  advances, 
and  changes  in  team  training  and  procedures  that  will  improve  the  proficiency  of 
the  personnel  staffing  Navy  teams. 

Conclusion  3:  Several  techniques  exist  that  could  make  immediate  improve¬ 
ments  in  the  training  of  team  members:  providing  objective  feedback  during  train¬ 
ing,  improving  the  match  between  training  and  job  requirements,  and  expanding 
the  use  of  computer-based  instruction.  Technology-transfer  studies  should  be  un¬ 
dertaken  to  determine  whether  and  how  these  improvements  should  be  implement¬ 
ed  in  operational  environments. 

Conclusion  4:  New,  longer-term,  basic  research  should  involve  the  study  of 
decisionmaking  teams  in  realistic  task  environments.  This  would  not  necessarily 
require  continual  direct  contact  between  researchers  and  Navy  teams,  but  re¬ 
searchers  should  at  least  utilize  simulated  task  environments  similar  to  those  in  the 
operational  Navy.  This  will  require  observation  of  and  familiarization  with  teams 
in  operational  settings. 

Conclusion  5:  The  more  promising  lines  of  research  will  require  multivariate 
studies  of  team  performance  over  time.  Such  studies  provide  an  integrated  ap¬ 
proach  to  team  research  that  is  not  adequately  achieved  by  narrower  studies  of 
individual  dimensions  of  team  performance.  Research  should  include  development 
of  methods  for  simulating  operational  task  performance  in  the  laboratory,  con¬ 
trolled  experimental  studies  of  team  performance,  and  studies  of  performance 
evolution. 

Conclusion  6:  Several  approaches  to  improving  team  performance  appear  to 
be  promising  areas  for  research: 

1.  Improving  understanding  of  the  task  by  analyzing  performance  require¬ 
ments.  In  particular,  research  should  investigate  the  criteria  for  assessing 
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performance,  the  knowledge  required  for  skilled  performance,  the  vari¬ 
ables  that  influence  or  limit  task  performance,  and  the  relationship  be¬ 
tween  individual  and  team  performance. 

2.  Improving  selection  of  team  members  by  matching  individual  skills  and 
abilities  to  task  demands. 

3.  Improving  training  and  exercising  of  both  individuals  and  teams  by  deve¬ 
loping  standard  training  methodologies,  feedback  procedures,  personal¬ 
ized  instruction  by  computer,  and  training  games. 

4.  Compensating  for  the  disruptive  effects  of  turnover  by  improving  training 
efficiency,  improving  socialization  and  integration  of  new  members  into 
the  team,  and  reducing  attrition. 

5.  Improving  the  organization  of  the  team  by  altering  standard  operating 
procedures,  communications  networks  and  protocols,  or  decisionmaking 
processes. 

6.  Developing  improved  man-machine  systems  and  aids  for  task  perfor¬ 
mance. 

Conclusion  7:  This  multivariate,  multidisciplinary  research  requires  one  or 
more  Team  Performance  Research  Centers  that  have: 

1.  A  laboratory  facility  for  experimental  studies  of  team  performance. 

2.  A  system  for  representing  and  controlling  combat-like  situations  in  which 
teams  perform  their  tasks.  This  system  must  include  models  of  the  envi¬ 
ronment,  models  of  the  effects  of  the  team’s  and  enemy  actions,  mecha¬ 
nisms  for  dynamic  scenario  control,  an  interface  between  the  team  and  the 
environmental  model,  and  communications  links  among  team  members. 

3.  Advanced  computer  hardware  and  software  that  can  simulate  important 
functional  characteristics  of  operational  equipment  and  communications. 
Requisite  hardware  includes  multi-user  time-shared  processors  and/or  a 
network  of  small  processors,  and  terminals  with  graphics  display  capabili¬ 
ties. 

Conclusion  8:  Designing  a  Team  Research  Center  and  conducting  the  re¬ 
search  will  require  an  interdisciplinary  team  of  scientists.  This  team  must  include 
experts  in  cognitive  psychology,  social  psychology,  human  factors,  software  and 
systems  design,  artificial  intelligence,  simulation,  gaming,  and  instruction. 

Conclusion  9:  Basic  research  supported  by  the  program  should  be  comple¬ 
mented  by  policy  studies  to  determine  the  potential  impact  and  applicability  of 
research  results  in  the  operating  environment.  These  studies  should  include: 

1.  An  analysis  of  future  Navy  requirements  and  personnel  resources  to  de¬ 
termine  (a)  which  teams  are  most  critical  to  operational  effectiveness  and 
which  can  realize  significant  performance  improvements  from  research, 
and  (b)  how  procedures  and  training  will  need  to  be  adapted  to  the  abilities 
of  future  personnel.  Current  team  problems  and  obstacles  to  improving 
performance  may  change  rapidly  as  the  character  and  technology  of  the 
Navy  changes.  Appropriate  targets  toward  which  to  aim  research  thrusts 
must  be  identified. 

2.  An  analysis  of  the  barriers  to  translating  research  results  into  changes  in 
policy  and  procedures  in  operational  environments. 
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Conclusion  10:  To  increase  the  impact  and  effectiveness  of  the  research  pro¬ 
gram,  ONR  should  include  the  targeted  user  and  applied  research  communities  in 
the  program.  Representatives  of  these  communities  might  serve  several  functions, 
including: 

1.  Assisting  in  the  identification  of  critical  teams  for  study  and  in  the  deter¬ 
mination  of  highly  skilled  examples  of  such  teams. 

2.  Suggesting  problems  or  bottlenecks  in  improving  team  performance. 

3.  Facilitating  the  transition  of  research  results  into  applied  development 
efforts. 

4.  Providing  researchers  with  access  to  facilities  and  resources  in  operation¬ 
al  or  training  environments. 

Conclusion  11:  Implementation  of  the  proposed  program  should  proceed  in 
stages: 

1.  Immediate  initiation  of  a  study  to  select  the  task  domain(s)  for  inclusion 
in  the  program  and  to  specify  the  important  task  characteristics.  In  addi¬ 
tion,  the  study  should  identify  Navy  personnel  and  facilities  willing  to 
cooperate  with  research  contractors  in  the  program.  These  studies  could 
be  carried  out  in  the  current  fiscal  year. 

2.  Identification  of  research  institutions  willing  and  able  to  undertake  a 
long-term  laboratory  development  and  interdisciplinary  research  effort. 

3.  Initiation  of  activities  to  coordinate  researchers  and  evaluate  research 
results,  including 

a.  Formation  of  a  steering  committee  to  monitor  the  research  program. 

b.  Sponsorship  of  a  workshop  to  familiarize  contractors  with  the  pro¬ 
gram  structure,  task  domain,  performance  specifications,  and  oper¬ 
ational  equipment. 

c.  Insuring  ARPANET  access  for  all  contractors. 

4.  Initial  research  focused  on  the  development  of  laboratory  facilities,  simu¬ 
lations,  and  games. 

5.  A  program  management  decision,  approximately  three  years  into  the 
program,  about  how  to  continue.  The  decision  should  be  based  on  contrac¬ 
tors’  progress  in  constructing  laboratory  facilities,  the  quality  of  the  task 
representations,  initial  results  of  studies  using  the  laboratories,  and 
proposals  for  future  research.  Potential  decisions  include 

a.  Continued  funding  of  all  contractors  to  perform  additional  research. 

b.  Continued  and  possible  increased  funding  of  fewer  contractors. 

c.  Delaying  the  decision  because  no  contractor  has  assembled  the  desired 
facilities. 

d.  Redirecting  the  program  to  smaller  efforts  either  more  removed  from 
specific  applications  or  focused  on  a  smaller  set  of  research  issues,  as 
in  current  ONR  6.1  programs. 

e.  Terminating  the  program. 

6.  Continuation  of  major  research  efforts  for  two  to  seven  years. 


4.  MANAGEMENT  POLICIES  AND  ACTIVITIES 


The  program  proposed  in  the  previous  sections  is  characterized  by  the  close 
coupling  of  field  research,  the  development  of  experimental  laboratory  facilities, 
and  basic  research  activities.  This  reflects  a  belief  that  the  research  environment 
should  functionally  resemble  a  targeted  operational  environment.  Such  coupling 
should  also  facilitate  the  transition  of  6.1  research  results  into  applied  development 
efforts. 

Another  important  feature  of  the  program  is  that  the  recommended  domain  is 
represented  by  critically  important  teams  in  all  three  Services.  Thus,  the  proposed 
research  program  could  be  of  significant  benefit  to  the  entire  defense  community. 
However,  the  ultimate  utility  of  the  research  results  for  improving  operational 
team  performance  will  depend  largely  on 

1.  Anticipating  the  nature  and  needs  of  military  personnel,  teams,  and  sys¬ 
tems  10  to  15  years  from  now. 

2.  Communicating  to  potential  contractors  the  ultimate  applied  objectives  of 
the  research  as  well  as  the  immediate  objectives  of  the  basic  research 
program. 

3.  Identifying  the  best  research  projects  and  researchers. 

4.  Attracting  these  high-quality  researchers  to  the  research  program. 

5.  Communicating  the  benefits  of  the  research  enterprise  to  potential  users 
in  the  Navy  and  other  Services. 

6.  Participating  in  the  translation  of  successful  basic  research  into  applied 
systems  and  contexts. 

These  activities  represent  refinements  of  the  program  objectives  outlined  in 
Section  2.  ONR  management  should  include  these  activities  within  the  administra¬ 
tion  of  the  research  program.  We  suggest  below  a  set  of  detailed  methods  for 
implementing  these  activities. 


4.1.  ORGANIZATION  OF  THE  RESEARCH  PROGRAM 

AROUND  PROJECTED  OPERATIONAL  NEEDS  OF  THE  1990s 

Navy  teams  and  their  training  requirements  undergo  rapid  and  continual 
change.  The  changes  include  the  composition,  skills,  and  term-expectancies  of  en¬ 
tering  Navy  personnel,  as  well  as  in  the  sophistication  and  complexity  of  equip¬ 
ment.  By  the  1990s,  significant  changes  may  occur  in  the  relationship  between 
humans  and  machinery,  introducing  new  training,  job-skill,  and  team-organization 
requirements.  The  prominent  and  critical  Navy  team  activities  of  the  1990s  will 
very  likely  be  different  from  those  currently  central  to  Navy  operations. 

If  research  begun  now  is  to  have  impact  on  Navy  operations  and  technologies 
of  the  1980s  and  1990s,  it  must  consider  the  requirements  of  the  future,  as  well  as 
the  present,  operational  Navy.  Thus,  one  of  the  first  activities  of  the  research 
program  should  be  a  policy  study  of  the  types  of  teams  that  will  be  required  to  make 
command  decisions  in  the  1990s,  what  the  skills  of  available  personnel  are  likely 
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to  be,  and  where  improvements  in  team  performance  would  have  the  greatest 
impact  on  Navy  operations. 


4.2.  FOCUSING  PROGRAM  RESOURCES  IN  A  PARTICULAR 
TASK  DOMAIN 

The  range  of  Navy  teams  and  team  problems  far  exceeds  the  set  of  issues  that 
the  research  program  could  reasonably  support.  To  produce  results  with  the  great¬ 
est  impact  on  the  operational  Navy,  the  program  should  focus  attention  on  im¬ 
provement  of  the  operation  of  information-processing,  decisionmaking  teams, 
which  are  critical  to  successful  Navy  functioning. 

To  guarantee  the  applicability  of  research  results  obtained  in  the  laboratory  to 
the  solution  of  Navy  problems,  contractors  should  be  required  to  simulate  actual 
operational  teams.  This  requirement  may  entail  a  significant  development  effort  to 
design  and  create  a  team  performance  laboratory,  but  such  a  laboratory  will  facili¬ 
tate  technology  transfer  to  applied  research  programs. 


4.3.  PROMOTION  OF  COHESIVENESS  AND  SYNERGY 
AMONG  THE  RESEARCH-PROGRAM  PROJECTS 

To  obtain  maximum  leverage  from  the  research  program,  progress  should 
derive  not  only  from  the  accumulation  of  results  from  individual  projects  but  from 
synergy  and  cross-fertilization  of  ideas  among  contractors.  To  promote  close  ties 
among  researchers,  all  projects  should  be  constrained  by  a  common  set  of  target 
teams,  problems,  and  ultimate  operational  objectives.  Individual  researchers  or 
research  projects  could  investigate  different  but  complementary  components  of  a 
larger  problem,  while  sharing  a  common  task  environment  and  set  of  research 
objectives.  Cohesion  in  the  research  program  could  be  encouraged  by  the  following 
recommendations: 

1.  Structure  the  research  program  around  a  small  number  of  large-budget, 
interdisciplinary  projects.  Each  project  should  include  a  number  of  individual  or 
subproject  research  activities,  such  as  the  design  and  implementation  of  laboratory 
hardware  and  software,  the  design  and  execution  of  social-  and  cognitive-psychol¬ 
ogy  experiments,  task  simulations,  games,  and  the  development  and  testing  of 
training  methods.  Each  project  would  thus  utilize  a  multipurpose  laboratory  facili¬ 
ty  and  researchers  from  several  discipline  areas.  The  scale  of  the  individual  re¬ 
search  efforts  and  the  existence  of  team  observation  and  experimentation  laborato¬ 
ries  would  give  the  research  program  high  visibility. 

2.  Form  a  steering  committee  to  select,  coordinate,  and  administer  the  projects 
in  the  research  program.  The  committee’s  functions  should  include  reviewing 
proposals,  determining  the  relevance  of  specific  proposals  and  projects  to  the  over¬ 
all  goals  of  the  program,  coordinating  and  promoting  communication  among  the 
contractors  and  operational  personnel  (see  4.4  and  4.5  below),  and  periodically 
evaluating  the  progress  of  the  various  research  projects.  A  representative  of  the 
research  community  could  organize  and  manage  the  activities  of  the  steering  com¬ 
mittee. 
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3.  Conduct  annual  workshops  at  which  contractors  present  research  results 
and  representatives  of  the  applied  research  and  operational  branches  of  the  Navy 
discuss  their  current  activities  and  problems. 

4.  Establish  a  clearinghouse  for  publications  produced  by  contractors  to  ensure 
wide  distribution  and  availability  of  research  reports. 

5.  Provide  access  to  the  ARPANET  for  all  contractors  in  order  to  facilitate 
communications  and  resource-sharing  within  the  program. 


4.4.  THE  DEVELOPMENT  OP  A  NATIONAL  COMMUNITY  OF 
TEAM  RESEARCHERS 

To  develop  an  outstanding  national  team  performance  research  program,  ONR 
should  attempt  to  promote  interest  in  team  problems  among  researchers  in  the 
fields  of  psychology,  computer  science,  management  science,  and  education.  While 
a  considerable  body  of  research  on  teams  has  accumulated  over  the  past  20  years, 
that  research  has,  by  and  large,  been  isolated  within  disciplines,  and  little  of  it  has 
had  any  direct  relevance  to  real  Navy  team  problems.  Consequently,  ONR  should 
attempt  to  generate,  support,  and  reward  interest  in  Navy  team  problems  in  order 
to  attract  researchers  to  the  program. 

The  problems  embraced  by  the  research  program  must  be  scientifically  tracta¬ 
ble,  interesting,  and  intellectually  stimulating.  To  guarantee  this,  the  research 
problems  and  the  applicability  of  research  results  should  extend  beyond  the  par¬ 
ticular  task  domains  under  investigation. 

In  addition,  the  research  issues  must  be  communicated  to  and  publicized  in  the 
research  community.  Researchers  must  be  attracted  to  the  program  and  their 
interest  must  be  sustained.  ONR  might  consider  sponsoring  an  annual  conference 
or  workshop,  such  as  the  one  held  at  Rand  in  November  1979,  for  team  performance 
researchers;  awarding  an  annual  prize  for  the  most  outstanding  publication  or 
other  contribution  to  the  field;  providing  fellowship  funds  or  allowances  to  support 
the  training  of  graduate  students  in  research  on  teams;  and  providing  funds  for  the 
establishment  of  a  research  institute  to  operate  in  conjunction  with  one  or  more  of 
the  program-supported  laboratory  facilities.  Such  an  institute  might  support  the 
post-doctoral  or  sabbatic  research  of  professionals  interested  in  beginning  research 
on  teams  and  could  sponsor  symposia,  colloquia,  or  workshops  on  various  aspects 
of  team  performance. 


4.5.  ACTIVE  PARTICIPATION  IN  TRANSFER  OF  BASIC  RESEARCH 
RESULTS  TO  APPLIED  DEVELOPMENT  PROGRAMS 

Many  useful  results  from  ONR-sponsored  research  have  not  been  translated 
into  6.2-6.4  development  programs.  For  results  produced  in  the  team  performance 
program  to  have  visibility  and  impact,  they  must  be  transferred  to  and  tested  in 
operational  environments.  To  facilitate  this  process,  ONR  should  invite  key  individ¬ 
uals  from  other  R&D  areas  and  from  the  operational  and  training  commands  to 
participate  in  the  research  program.  As  mentioned  above,  such  individuals  might 
participate  in  annual  contractors’  meetings  or  research  conferences,  or  they  might 
serve  on  the  program  steering  committee.  Such  participation  would  help  to  ensure 
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the  proper  research  focus  and  to  solidify  the  investment  of  these  communities  in 
the  program.  In  addition,  ONR  should  attempt  to  involve  basic  researchers  in  the 
development  process  to  guide  the  implementation  of  research  results  in  initial 
prototype  demonstrations. 


4.6.  CONTINUING  EVALUATION  OF  RESEARCH  PROGRESS 

The  progress  of  program  researchers  should  be  critically  evaluated  periodical¬ 
ly.  Contractors  should  initially  develop  five-year  research  plans,  but  continuation 
of  funding  should  depend  on  contractors’  abilities  to  reach  research  milestones  and 
on  their  progress  in  comparison  with  other  contractors.  Research  teams  should,  to 
some  extent,  be  in  competition  with  each  other,  at  least  during  the  early  stages  of 
laboratory  and  simulation  development. 

The  program  should  also  allow  flexibility  for  altering  research  directions  as 
dictated  by  the  results  of  early  policy  studies.  For  example,  if  studies  uncover  a 
particular  type  of  team  or  function  critical  to  Navy  operations  of  the  1990s,  existing 
or  new  projects  might  be  directed  toward  investigating  issues  particular  to  this 
team  or  function. 


4.7.  ESTABLISHMENT  OF  A  TRI-SERVICE  RESEARCH  PROGRAM 

The  research  issues  in  the  proposed  research  program  transcend  the  interests 
of  the  U.S.  Navy.  Information-handling  teams  that  assess  situations,  plan,  make 
decisions,  and  allocate  resources  are  ubiquitous  in  the  Air  Force  and  the  Army  as 
well.  The  Tactical  Air  Control  Systems  of  the  U.S.  Air  Force  and  the  Tactical 
Operations  Centers  of  the  U.S.  Army  resemble  Navy  combat  information  centers, 
so  many  research  results  in  this  program  will  apply  to  activities  in  the  other 
Services.  A  major  tri-Service  research  program  would  offer  the  opportunity  for 
increased  leverage  in  many  aspects  of  the  program,  including  its  cohesiveness  and 
synergy,  the  development  of  a  community  of  team  researchers,  the  transfer  of 
applicable  technology,  and  the  overall  cost  of  facilities.  ONR  should  lead  the  effort 
to  develop  a  such  a  tri-Service  program  in  team  performance. 

At  a  minimum,  ONR  should  propose  to  the  other  Services  a  time-phased  multi- 
Service  entry  and  commitment  to  research  activities.  The  suggested  steering  com¬ 
mittee  for  the  ONR  research  program  should  therefore  include  members  of  the 
other  military  Services  with  responsibility  for  establishing  interfaces  with  their 
respective  research  offices  as  a  move  toward  an  integrated  and  jointly  funded 
program. 


5.  RESEARCH  ACTIVITIES 


This  section  outlines  the  set  of  R&D  activities  that  should  be  supported  under 
the  Team  Performance  program.  The  set  comprises  26  distinct  but  interrelated 
research  activities,  which  fall  into  the  following  eight  broad  areas: 

1.  Provide  policy  recommendations  that  will  maximize  the  impact  of  the 

team  performance  research  (Sections  5.1.1-5.1.4). 

2.  Develop  task  specifications  and  models  (Section  5.2). 

3.  Develop  facilities  for  task  simulation  and  experimentation  (Sections  5.3. 1- 

5.3.3) . 

4.  Develop  improved  performance  models  and  evaluation  metrics  (Sections 

5.4.1- 5.4.4). 

5.  Investigate  the  effects  of  team  synergy  and  personnel  turnover  (Sections 

5.5.1- 5.5.2). 

6.  Improve  the  organization  of  teams  (Sections  5.6.1-5.6.3). 

7.  Improve  the  training  of  teams  and  team  members  (Sections  5.7.1-5.7.6). 

8.  Improve  man-machine  systems  for  task  performance  (Sections  5.8.1- 

5.8.3) . 

In  each  of  these  areas,  we  suggest  several  promising  topics  for  research.  For 
each  activity,  we  describe  the  problem  that  motivates  our  recommendation  and  the 
important  issues  to  be  addressed. 


5.1.  POLICY  ASSESSMENTS  OF  NAVY  NEEDS 

To  assess  the  potential  impact  of  basic  research,  ONR  must  undertake  a  set  of 
policy  studies  to  identify  (1)  those  issues  related  to  team  effectiveness  that  are  likely 
to  remain  critical  over  15  years  of  strategic  and  technological  change  and  (2)  those 
approaches  to  team  improvement  that,  if  successful,  have  some  likelihood  of  being 
adopted  in  the  operating  environment.  We  recommend  four  specific  projects  below. 

5.1.1.  Navy  Teams  of  the  Future1 

A  research  program  on  team  training  and  performance  should  support  long¬ 
term  Navy  requirements  in  three  ways: 

1.  By  determining  the  types  of  team  activities  and  functions  that  will  be 
required  to  operate  threat-evaluation  and  decisionmaking  systems 
projected  for  procurement  over  the  next  two  decades. 

2.  By  directing  research  toward  these  anticipated  team  functions  and  activi¬ 
ties. 

3.  By  providing  guidance,  based  on  results  of  program  research,  early 
enough  that  adequate  team  training  and  evaluation  capabilities  and  hu- 


Written  by  Milton  Weiner. 


21 


man-factors  design  principles  can  be  incorporated  into  the  design  of  new 
equipment. 

This  proposed  effort  would  focus  on  the  first  two  items:  defining  the  team 
functions  and  activities  needed  to  support  advanced  systems,  and  identifying  re¬ 
search  program  requirements  to  produce  relevant  results.  There  are  a  number  of 
sources  of  data  on  anticipated  Navy  needs  for  personnel  and  for  particular  skills, 
as  well  as  sources  that  categorize  naval  personnel  functions  and  activities,  e.g.,  the 
Navy  Manpower  Mobilization  System  (NAMMOS).  Other  sources  of  Navy  data  on 
longer-range  planning  needs  for  manpower  and  skills  for  new  systems  would  also 
contribute  to  the  investigation  of  future  Navy  teams.  Procurement  decisions  may 
have  already  generated  requirements  for  team  skills  several  years  into  the  future, 
but  there  is  no  documentation  or  longer-term  projection  of  anticipated  needs  for 
different  types  of  teams  or  for  the  skills  associated  with  these  teams.  Such  a  projec¬ 
tion  could  assist  in  determining  the  types  of  team  activities  and  functions  for  which 
research  would  have  the  greatest  utility  and  it  could  be  used  to  establish  policy 
guidelines  for  the  research  program  in  terms  of  (1)  what  team  functions  or  activities 
seem  critical,  (2)  which  research  questions  have  highest  priority,  (3)  what  research 
strategy  is  indicated,  and  (4)  how  the  research  results  are  to  be  used. 

This  effort  would  require  no  more  than  a  9-  to  12-month  study  by  two  or  three 
analysts,  since  a  great  deal  of  data  may  already  be  available.  The  major  effort  would 
focus  on  the  development  of  the  research  guidelines  implied  by  these  data.  Such 
a  study  would  assure  that  the  research  program  is  compatible  with,  and  perhaps 
could  contribute  to,  the  design  and  operation  of  advanced  systems. 

5.1.2.  Organizational  Factors  in  Introducing  Innovation2 

The  frequent  rotation  of  operational  command  personnel  produces  disincen¬ 
tives  to  undertake  time-  and  resource-consuming  programs  of  innovation.  Such 
resistance  may  discourage  the  development  of  more  effective  technologies  for  train¬ 
ing  or  the  adoption  of  current  technologies.  The  Task  Force  on  Training  Technology 
recommended  in  1976  that  the  DoD  must  change  management  policies  to  provide 
greater  incentives  for  the  development  and  adoption  of  training  technologies  (De¬ 
fense  Science  Board,  1976).  In  line  with  this  recommendation,  the  research  program 
should  include  studies  focused  on  understanding  and  improving  the  adoption  pro¬ 
cess.  The  two  principal  activities  of  these  studies  are  described  below. 

1.  Understanding  the  Adoption  Process.  Several  factors  inhibit  the  adoption 
of  new  technologies  and  methods  for  tactical  decisionmaking  teams.  Since  it  is 
difficult  to  define  precisely  the  activities  of  such  teams,  it  may  not  be  clear  what 
innovations  are  necessary  or  how  they  should  be  evaluated  (Kaplan  and  Barber, 
1979).  Even  when  requirements  for  team  performance  can  be  established,  innova¬ 
tion  may  be  too  technically  demanding,  costly,  or  inconvenient  (Hall  and  Rizzo, 
1975).  Finally,  the  receiving  organization  may  resist  some  innovations  because  it 
fears  that  they  will  result  in  other,  unwanted  changes.  For  example,  a  training  unit 
may  be  reluctant  to  urge  operating  organizations  to  evaluate  the  adequacy  of  its 
graduates  because  resulting  criticisms  may  dictate  changes  in  training  procedures. 
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Resistance  to  change  is  especially  strong  where  tradition  is  highly  valued  for  its 
own  sake — and  some  Navy  units  may  be  strongly  bound  by  tradition. 

Problems  of  transferring  successful  research  results  to  operating  environments 
have  been  studied  extensively  in  several  contexts,  including  education  (Carpenter- 
Huffman,  Hall,  and  Sumner,  1974;  Carlson,  1975)  and  manufacturing  (Mansfield, 
1961).  Using  a  multivariate  analysis  of  Teacher  Corps  programs,  Corwin  (1973) 
derived  a  profile  of  a  "social  system  likely  to  be  highly  responsive  to  deliberate 
efforts  to  change  it.”  A  number  of  characteristics  of  this  profile,  including  as  central¬ 
ization  of  authority  and  a  clear-cut  division  of  labor,  apply  to  the  Navy.  Finally, 
Berman  and  McLaughlin  (1978)  proposed  that  successful  innovation  in  education 
requires  a  process  of  mutual  adaptation  in  which  both  the  innovator  and  the 
recipient  of  the  innovation  are  active  partners. 

Similar  research  should  investigate  these  issues  in  the  Navy  organization, 
possibly  combining  the  case  study  and  multivariate  analytical  approaches  adopted 
by  previous  analysts.  If  successful,  this  work  will  reinforce  the  entire  research 
program  by  articulating  constraints  on  and  methods  for  translating  research  into 
improvements  in  Navy  operations. 

2.  Establishing  the  Value  of  Innovation.  Since  innovation  can  be  costly  in 
itself  and  can  levy  additional  costs  on  the  units  that  adopt  it,  a  second  research 
effort  should  develop  methodologies  for  establishing  the  costs  and  benefits  to  the 
Navy  of  any  proposed  innovation.  These  include  procurement  and  implementation 
costs,  organization  disruption,  and  increments  in  resulting  operating  performance. 
Recent  studies  of  this  nature  conducted  for  the  Air  Force  have  produced  conclusive 
assessments  of  alternative  training  courses  for  electronic  and  avionics  equipment 
repairmen  (Air  Training  Command,  1968,  1970;  Carpenter-Huffman  and  Rost ker, 
1976). 

Additional  research  to  develop  methodologies  for  establishing  the  value  of 
innovations  could  begin  with  small-scale  demonstrations  in  the  operating  environ¬ 
ment.  Their  results  could  then  be  used  to  help  determine  the  Navy-wide  applicabili¬ 
ty  of  the  innovation  and  its  overall  cost  and  value. 

The  first  step  will  be  to  choose  a  promising  innovation  whose  effects  may  be 
demonstrated  in  the  near  future.  Likely  candidates  include  (1)  improved  feedback 
in  team  training  and  shipboard  exercising,  (2)  improved  matching  of  training  in 
individual  skills  to  team  operating  requirements,  and  (3)  computer-directed  individ¬ 
ual  drill  and  practice  for  potential  team  members. 

Demonstrations  of  this  kind  may  require  a  year  or  more  to  design,  implement, 
and  evaluate.  The  design  phase  is  the  most  crucial  because  it  must  ensure  that  the 
obtained  data  will  permit  assessment  of  the  cost  of  the  innovation  and  its  value  in 
operating  units.  Much  of  this  work  should  probably  be  conducted  by  Navy  oper¬ 
ations  and  research  personnel,  with  contractor  assistance,  in  existing  Navy  facili¬ 
ties. 

It  will  probably  not  be  difficult  to  demonstrate  positive  value  for  the  innovation 
chosen  for  study.  However,  development  of  generalized  methodologies  for  estab¬ 
lishing  value  will  be  more  demanding.  Nevertheless,  the  potential  payoff  of  this 
study  is  sufficiently  high  to  warrant  the  risk  of  failure. 
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5.1.3.  Current  Technologies  for  Improving  Navy  Teams3 

During  the  past  decade,  several  advances  have  created  opportunities  to  im¬ 
prove  training,  most  notably  the  development  of  systematic  techniques  to  (1)  im¬ 
prove  feedback  to  the  performer  during  training  and  exercising,  (2)  improve  the 
match  between  training  and  requirements  for  job  performance,  and  (3)  improve 
training  through  the  expanded  use  of  computer-based  techniques.  The  Services, 
however,  have  not  adopted  these  advances  widely.  Therefore,  new  studies  should 
develop  plans  and  specific  methods  for  introducing  the  following  improvements  into 
operational  environments. 

1.  Improved  Feedback.  As  discussed  in  Section  5.7.6,  previous  experimental 
research  has  established  several  principles  governing  the  relationship  between 
feedback  and  performance.  Currently,  feedback  in  Navy  team  training  does  not 
adhere  to  these  principles;  it  is  generally  haphazard  and  subjective,  varying  across 
team  roles,  insti  actors,  and  training  installations  (Hall  and  Rizzo,  1975).  With  little 
additional  cost  or  research,  it  may  be  possible  to  improve  and  systematize  feedback 
in  such  training  situations. 

2.  Improved  Match  Between  Training  and  Job  Requirements.  Hall  and  Rizzo 
(1975),  among  others,  have  pointed  out  the  frequent  discrepancies  between  the 
content  of  training  and  the  requirements  for  job  performance.  Failure  to  provide 
adequate  training  places  added  burdens  on  operating  units  and  can  ultimately 
degrade  combat  capability. 

A  systematic  procedure.  Instructional  System  Development  (ISD),  has  been 
developed  to  deal  with  the  problem,  and  the  Navy  has  recently  established  Instruc¬ 
tional  Program  Development  Centers  to  facilitate  application  of  ISD  (Vineberg  and 
Joyner,  1980).  These  centers  are  attempting  to  document  the  skills  required  for 
every  Navy  job  rating,  but  they  are  currently  staffed  primarily  by  training  person¬ 
nel  rather  than  persons  with  direct  experience  in  team  operation. 

Studies  in  this  area  should  focus  on  strengthening  participation  of  operational 
personnel  in  the  ISD  process.  Policy  analysts  and  management  specialists  should 
test  the  efficacy  of  different  approaches  to  improving  cooperation  between  trainers 
and  operators,  ranging  from  workshops  to  formal  organizational  structures.  Ve¬ 
hicles  for  these  tests  will  arise  in  the  normal  course  of  revision  of  training  for 
selected  teams.  Researchers  could  observe  the  efficacy  of  a  variety  of  approaches 
and  procedures  to  select  the  most  promising  for  implementation. 

3.  Application  of  Existing  Computer-Based  Techniques.  Two  techniques  in 
computer-based  instruction  could  immediately  improve  team  training:  drill-and- 
practice  and  simulation.  (It  is  not  clear  that  "frame-type”  CAI,  a  third  technique 
now  available  on  many  commercial  systems,  is  more  effective  than  conventional 
programmed  texts.) 

Drill  and  Practice.  Effective  training  requires  that  team  members  master  the 
factual  knowledge  required  to  perform  their  individual  jobs.  Adaptive  systems  for 
drill-and-practice  based  on  psychological  learning  theory  train  learners  faster  than 
non-adaptive  drill-and-practice  systems,  result  in  better  long-term  retention  of 
knowledge,  and  enable  the  instructor  to  monitor  the  learners’  progress.  For  some 
knowledge  domains,  drill-and-practice  can  be  embedded  in  games  to  motivate  the 
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learner  and  thus  further  reduce  training  time.  Team  jobs  that  require  memorizing 
a  substantial  body  of  simple  factual  relationships  will  probably  benefit  most  from 
this  technology. 

The  Navy  Personnel  Research  and  Development  Center  in  San  Diego  is  deve¬ 
loping  an  experimental  prototype  drill-and-practice  system  for  teaching  tactical 
action  officers  the  characteristics  of  enemy  platforms.  Other  drill-and-practice  sys¬ 
tems  could  be  implemented  on  inexpensive,  personal  microcomputers,  placed  at 
several  locations  within  a  training  site. 

Simulations  for  Team  Practice.  Some  teams  have  inadequate  opportunities 
for  hands-on  practice  of  routine  procedures  or  have  no  chance  to  practice  emergen¬ 
cy  procedures,  particularly  when  the  procedures  are  expensive  or  dangerous.  This 
problem  could  be  alleviated  by  combining  operational  systems  and  general-purpose 
digital  computers  to  create  the  relevant  features  of  the  task  environment  and 
provide  functional  simulations. 

Simulations  have  several  advantages  over  operational  exercises.  They  permit 
creation  of  situations  that  are  difficult  to  arrange  in  the  real  world,  such  as  emer¬ 
gencies.  They  allow  systematic  manipulation  of  the  team  functions  to  enhance 
trainee  perception  of  critical  task  features.  And  they  can  store  performance  proto¬ 
cols  in  the  computer  for  replay  during  debriefing  and  critique.  Simulations  have 
been  used  successfully  in  a  number  of  task  domains.  The  Navy  Training  Equipment 
Center  at  Orlando  is  developing  prototype  simulators  that  rely  more  heavily  on 
general-purpose  computers  and  input/output  (I/O)  devices  (e.g.,  graphics)  for 
teaching  than  have  previous  simulators.  In  addition,  the  Navy  Air  Traffic  Control 
School  at  Memphis  Naval  Air  Station  trains  carrier  air  traffic  control  teams  using 
computer-based  systems  and  operational  radar  displays.  The  Army  Research  Insti¬ 
tute  at  Alexandria  is  developing  experimental  prototype  low-fidelity,  functional 
simulations  to  teach  a  range  of  individual  skills.  Finally,  the  Air  Force  is  developing 
simulators  for  training  maintenance  technicians. 

More  extensive  use  of  simulations  could  improve  training  in  other  domains  as 
well.  Past  development  of  simulations  for  training  seems  to  have  proceeded  most 
effectively  where  computer  technology  is  an  accepted  part  of  the  operational  envi¬ 
ronment,  as  in  aviation  and  air  traffic  control.  Expanded  application  of  this  technol¬ 
ogy  will  require  a  commitment  to  computer-based  training  and  a  systematic  effort 
to  identify  individual  and  team  tasks  for  which  such  techniques  can  be  used. 

5.1.4.  Causes  and  Effects  of  Personnel  Turnover  in  Navy  Teams4 

Fluctuations  in  team  membership  may  result  from  personnel  rotation,  attri¬ 
tion,  illness,  rotation  of  personnel  on  watch  stations,  and  so  on.  Turnover  in  the 
Navy  may  well  be  both  a  cause  and  a  consequence  of  low  team  performance  and 
morale.  Organizational  studies  of  turnover  and  its  consequences  are  needed  to 
determine  (1)  the  impact  of  turnover  on  morale  and  performance,  (2)  the  additional 
training  requirements  and  costs  introduced  by  turnover,  (3)  the  principal  sources 
and  causes  of  turnover,  (4)  the  feasibility  of  altering  policies  producing  turnover, 
and  (5)  implementable  policies  and  techniques  for  reducing  turnover  or  its  effects. 

While  a  few  organizational  studies  have  investigated  attrition  among  military 
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personnel  (see  review  by  Hand,  Griffeth,  and  Mobley,  1977)  little  previous  research 
has  attempted  such  a  broad  approach  to  the  study  of  turnover.  Experimental 
studies  of  teams  rarely  track  performance  over  time  or  in  cases  in  which  team 
composition  changes.  Thus,  several  new  policy  studies  could  be  valuable  in  assess¬ 
ing  the  influences  of  turnover  on  team  performance.  These  studies  are  outlined 
below. 

1.  Sources  of  Turnover.  One  line  of  research  might  investigate  the  principal 
sources  of  turnover  and  the  typical  timing  of  personnel  replacements.  Is  turnover 
due  primarily  to  personnel  rotation  and  attrition?  If  so,  how  much  lead  time  is 
available  to  prepare  for  team  member  departure?  Does  most  turnover,  permanent 
(attrition,  rotation)  and  temporary  (illness),  occur  while  ships  are  deployed  at  sea 
or  while  stationed  in  port?  The 

answers  to  these  questions  may  suggest  methods  for  better  anticipating  and 
compensating  for  personnel  turnover. 

2.  Reduction  of  Turnover.  Another  line  of  research  might  investigate  the 
extent  to  which  turnover  can  be  reduced.  This  would  require  (1)  a  study  of  why 
personnel  choose  to  leave  the  Navy,  and  (2)  fieldwork  to  determine  the  principal 
sources  of  dissatisfaction  among  Navy  personnel,  how  turnover  is  viewed  within 
the  Navy,  and  what  incentives  would  reduce  turnover.  Research  could  assess  the 
effects  of  the  Navy’s  personnel  rotation  policy  on  its  team  operations  and  the 
feasibility  of  alternative  personnel  management  policies.  The  consequences  of  such 
alternatives  for  training,  team  member  satisfaction,  and  overall  performance  could 
then  be  examined. 

For  example,  individuals  might  be  organized  into  teams  fairly  early  in  their 
duty  cycle  and  then  trained,  assigned,  and  housed  thereafter  as  a  team.  In  the  event 
of  personnel  losses  from  this  team,  the  entire  unit  might  then  be  replaced  by 
another  similarly  trained  team.  The  original  team  would  then  receive  and  integrate 
new  members  during  a  retraining  cycle.  Where  full  replacement  of  the  team  is  not 
practical,  two  options  remain.  A  team  with  only  slightly  diminished  membership 
could  continue  to  perform  short-handed.  Since  teams  would  presumably  be  trained 
to  operate  under  such  circumstances,  their  performance  might  be  quite  good,  even 
if  not  comparable  to  that  of  a  well-trained  team  operating  at  full  capacity.  On  the 
other  hand,  when  the  membership  of  the  team  has  been  drastically  reduced,  new 
assignments  could  be  made  as  is  done  today.  Such  a  scheme  is  only  illustrative  of 
the  types  of  alternative  policies  that  might  be  considered. 

3.  Reduction  of  the  Impact  of  Turnover.  Research  should  also  investigate 
methods  for  compensating  for  the  adverse  effects  of  turnover  within  existing  orga¬ 
nizational  constraints.  For  example,  it  may  be  useful  for  departing  team  members 
to  overlap  with  replacement  members  to  facilitate  indoctrination  and  socialization, 
or  to  provide  informal  training.  Section  5.5.2  proposes  basic  research  to  examine 
the  efficacy  of  other  compensatory  measures.  In  addition,  organizational  studies 
should  examine  the  feasibility  and  costs  of  introducing  these  new  procedures  into 
Navy  operations.  It  may  also  be  instructive  to  identify  teams  whose  performance 
is  not  degraded  by  turnover,  in  order  to  determine  task  characteristics  that  govern 
vulnerability  to  turnover.  This  might  suggest  techniques  for  structuring  teams  to 
reduce  the  negative  effects  of  personnel  turnover. 

Since  turnover  is  a  problem  that  affects  all  Navy  teams,  this  research  should 
have  a  high  payoff  and  wide  applicability.  It  would  entail  lengthy  observation  of 
several  different  teams  undergoing  personnel  change,  studies  of  organizational 
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policies,  and  the  widespread  collection  of  survey  data,  and  close  contact  with  vari¬ 
ous  Navy  personnel  and  installations  would  be  required.  The  project  staff  would 
have  to  provide  expertise  in  policy  analysis,  military  operations,  social  and  organi¬ 
zational  psychology,  and  survey  research. 


5.2.  DEVELOPMENT  OP  A  TASK  SPECIFICATION  FOR 

TACTICAL  DECISIONMAKING5 

The  military  Services  use  many  information-processing,  decisionmaking  teams, 
comprising  many  individuals,  many  diverse  and  difficult  functions,  and  various 
types  of  equipment.  In  operational  environments,  the  scope  of  a  team  is  not  clearly 
defined  but  may  vary  depending  upon  the  desired  degree  of  functional  aggregation. 
For  example,  an  ASW  team  comprises  the  ship’s  TAO,  the  anti-submarine  air 
controller  (ASAC),  the  UBP  for  weapons  deployment,  several  sonar  operators  and 
plotters,  helicopter  crews,  and  ASW  air  crews.  Similar  teams  of  individuals  monitor 
and  control  surface  warfare  and  anti-air  warfare  (AAW).  However,  the  CIC  "team” 
contains  individuals  from  each  of  these  other  teams,  as  well  as  the  AIC  and  the 
ship’s  weapons  coordinator  (SWC).  Thus  the  partitioning  of  these  individuals  into 
particular  teams  is  somewhat  arbitrary. 

To  identify  a  critical  team  or  teams  for  study,  research  should  investigate  a 
variety  of  teams,  perhaps  across  Services,  to  assess  the  similarities  and  differences 
among  them.  This  study  should  produce  several  results,  including: 

1.  The  selection  and  definition  of  a  domain  to  be  investigated  in  depth  by 
subsequent  research  .efforts.  The  selected  domain  may  be  either  a  particu¬ 
lar  real  military  team,  two  or  more  teams  that  are  highly  similar  in 
function  and  resources,  or  an  artificial  team  or  "game”  that  closely  resem¬ 
bles  one  or  more  actual  teams. 

2.  The  documentation  of  the  functional  specifications  and  information-proc¬ 
essing  characteristics  of  the  task  and  task  equipment  for  use  in  designing 
laboratory  facilities  and  subsequent  team  research.  This  will  require  the 
specification  of  the  inputs  (sensor  data,  environmental  conditions,  enemy 
actions,  action  outcomes,  messages)  and  outputs  (console  operations, 
plans,  decisions,  weapons  and  equipment  deployment,  messages)  for  each 
team  member’s  task. 

3.  The  characterization  of  typical  scenarios  for  team  performance. 

4.  The  documentation  of  relevant  Navy  resources  (e.g.,  trainers,  simulators, 
software,  task  and/or  training  experts)  that  are  potentially  available  and/ 
or  training  experts)  for  use  the  study  of  the  selected  team(s). 

This  work  should  be  undertaken  as  the  first  activity  of  contractors  in  the 
research  program.  Adequate  analysis  and  documentation  of  such  tasks  and  facili¬ 
ties  will  require  expertise  in  military  operations,  engineering,  computer  science, 
and  cognitive  psychology. 
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5.3.  DEVELOPMENT  OF  FACILITIES  FOR  TASK 
SIMULATION  AND  EXPERIMENTATION 

Research  on  the  training  and  performance  of  decisionmaking  teams  will  re¬ 
quire  laboratory  facilities  for  the  observation  of  these  teams  under  controlled 
conditions.  Several  problems  must  be  addressed  in  the  development  of  research 
facilities  to  support  realistic  task  simulations.  This  section  addresses  the  major 
tasks  required  for  that  development. 

5.3.1.  Environmental  Simulations  for  Mock  Operational 
Equipment8 

The  use  of  simulator-driven  exercises  would  improve  much  current  Navy  train¬ 
ing  and  team  performance.  Training  simulators  embody  two  components:  an  envi¬ 
ronmental  simulator  and  mock  operational  equipment.  The  simulator  provides  a 
source  of  data  and  stimuli  for  the  decisionmakers  and  then  reacts  to  their  actions 
in  a  manner  that  mimics  the  actual  combat  environment.  The  mock  operational 
equipment  permits  sensory-motor  inputs  and  outputs  similar  to  those  encountered 
in  actual  operations.  Ordinarily,  simulators  are  designed  exclusively  for  use  with 
special  training  equipment,  but  we  see  an  opportunity  to  separate  them  so  that  the 
simulators  can  support  continuing  learning  in  operational  environments.  These 
environmental  simulators  could  then  support  laboratory  studies  of  team  perfor¬ 
mance. 

For  many  reasons,  it  would  be  advantageous  for  simulated  training  experience 
to  be  distributed  more  widely  than  current  practices  permit.  For  example,  to  in¬ 
crease  vigilance,  motivation,  and  skills,  individual  operators  of  operational  equip¬ 
ment  could  train  or  practice  on  the  job  with  simulated  team  problems  during  idle 
or  low-stress  periods.  This  would  require  operational  equipment  that  could  accept 
simulated  events  or  could  execute  such  simulations  itself. 

Some  specific  topics  for  research  supporting  these  objectives  follow. 

1.  Design  of  a  Hardware/Software  Configuration  for  Laboratory  Simulations. 
Initial  work  should  focus  on  the  development  of  experimental  facilities  for  task 
simulation.  Each  contractor  should  choose  an  advanced  set  of  hardware  and  soft¬ 
ware  capabilities  that  can  (1)  support  the  environmental  simulation  tasks  of  labora¬ 
tory  exercises,  (2)  provide  a  basis  for  simulation  archiving  and  sharing,  and  (3) 
exploit  favorable  economic  and  computer-science  trends  in  system  architecture  and 
languages.  These  trends  favor  high-level  languages  for  software  implementation 
(e.g.  SIMULA,  SMALLTALK,  DIRECTOR),  large  address-space  computer  main¬ 
frames  (at  least  24-bits),  interactive  file  systems  (as  in  TENEX  or  UNIX),  closely 
coupled  interactive  graphics  for  debugging  and  monitoring  simulations,  and  local 
networking  to  allow  environmental  simulations  and  operational  equipment  emula¬ 
tors  to  execute  on  different  machines. 

2.  Representation  of  the  Environment.  Research  should  develop  models  of  the 
peacetime  and  combat  environments  of  the  decisionmaking  team  under  study, 
using  a  simulated  source  of  inputs  and  reactions.  This  simulation  should  provide 
for  incremental  development  and,  if  possible,  variable  execution  speeds  in  ex¬ 
change  for  variable  precision  and  fidelity.  Emulators  for  operational  equipment 
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should  also  be  developed  that  reflect  the  principal  information  processing  functions 
of  future  equipment  (e.g.,  storage,  display,  and  analysis  of  intelligence  reports  and 
sensor  data).  Detailed  physical  characteristics  of  the  equipment  emulators,  such  as 
function  keys,  specific  visual  displays,  and  keyboard  layouts,  are  less  critical  and 
may  be  ignored  for  the  present. 

Most  simulators  model  the  uncertainty  of  the  real  world  poorly.  Because  teams 
will  be  trained  to  perform  against  an  artificial  environment,  the  simulation  should 
incorporate  appropriate  kinds  of  variability  and  should  possess  the  capability  to 
generate  surprising  events  (i.e.,  highly  unlikely  but  not  impossible  situations)  spon¬ 
taneously.  These  efforts  should  be  coordinated  with  those  addressing  the  problem 
of  scenario  generation  described  in  Section  5.3.2. 

3.  Coordination  of  Multiple  Data  Sources  for  Scenario  Control.  When  a  team 
participates  in  an  exercise,  the  systems  must  coordinate  and  control  the  simulated 
scenario  and  the  timings  of  remote  events.  Simulations  must  incorporate  a  control 
structure  for  such  multi-event  team  exercises.  The  inherent  limitations  of  distrib¬ 
uted  simulations  and  exercises  (e.g.,  bandwidth,  synchrony,  validity)  should  there¬ 
fore  also  be  assessed. 

4.  Linking  of  Simulators  with  Operational  Equipment.  Research  should  in¬ 
vestigate  a  suitable  technology  to  embed  simulators  directly  within  operational 
equipment  (such  as  operator  consoles)  or  a  methodology  for  coupling  modularized 
simulations  and  operational  equipment.  Later,  a  method  for  standalone  training  or 
combined  operational/simulated  tasks  should  be  developed.  Finally,  research 
should  develop  methods  for  exploiting  the  resident  simulator  as  part  of  a  built-in 
test  (BIT)  system.  The  BIT  system  in  each  operational  equipment  unit  would  con¬ 
tinuously  contrast  simulator-produced  data  with  actual  operational  I/O  to  monitor 
the  equipment’s  behavior  and  to  promote  the  early  detection  of  unexpected  situa¬ 
tions. 

This  research  requires  modern  computing  hardware  and  advanced  program¬ 
ming  languages.  It  requires  the  collaboration  of  individuals  with  expertise  in  the 
task  environments,  characteristics  of  the  operational  equipment,  interactive  sys¬ 
tems,  simulation  languages,  distributed  computing,  and  computer-controlled  mul¬ 
tiperson  gaming. 

This  research  will  help  to  provide  the  basis  for  a  team  research  laboratory,  and 
it  could  also  lead  to  a  much-needed  new  generation  of  systems  in  the  Navy  (and 
elsewhere).  Such  systems  could  be  used  either  for  operational  requirements  or  for 
simulator-driven  on-the-job  training.  Moreover,  improved  team  training  simula¬ 
tions  would  provide  increased  access  to  realistic,  challenging,  and  unpredictable 
scenarios. 

5.3.2.  Simulation  and  Gaming  Scenario  Development7 

The  simulation  of  the  task  environment  emphasizes  representing  that  environ¬ 
ment  in  a  manner  that  permits  flexible  world  modeling  (e.g..  incremental  develop¬ 
ment,  variable  executive  speeds,  the  introduction  of  uncertainty  and  surprise). 
These  qualities  ultimately  reflect  "scenarios,”  sequences  of  conditions  and  events 
that  might  occur  in  combat  situations.  Because  of  the  almost  infinite  variety  of 
potential  combat  scenarios,  new  research  must  develop  scenarios  that  are 
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1.  Appropriate  (provide  useful  training  on  the  variety  of  team  tasks), 

2.  Adequate  (challenge  the  trainees), 

3.  Realistic  (resemble  potential  combat  situations),  and 

4.  Flexible  (cover  a  variety  of  situations,  including  surprising  and  unlikely 
events). 

Virtually  every  large-scale  combat  simulation  or  war  game  requires  consider¬ 
able  preparation  of  the  input  events  in  order  to  create  the  task  environment. 
However,  there  is  little  sound  theoretical  basis  for  determining  which  scenarios  will 
provide  effective  training.  Generally,  the  emphasis  has  been  on  "realism,”  but  the 
relationship  of  particular  scenarios  to  training  objectives  or  outcomes  is  seldom 
explicit.  That  is,  there  is  seldom  an  "experimental  design”  for  the  scenario  or  the 
simulated  events  derived  from  it.  Thus  it  is  difficult  to  relate  the  results  of  the 
simulation  (the  dependent  variables)  to  the  scenario  inputs  (the  independent  vari¬ 
ables).  New  research  in  this  area  should  include  a  number  of  tasks: 

1.  Review  of  current  Navy  training  simulations  to  determine  the  relation¬ 
ships  among  scenario  inputs,  training  outputs/ performance  evaluations, 
and  training  objectives;  these  data  would  be  used  to  construct  a  prelimi¬ 
nary  model  of  these  relationships  and  to  assess  scenario  appropriateness, 
adequacy,  realism,  and  flexibility. 

2.  Identification  of  scenario  attributes  that  are  useful  in  a  variety  of  team 
experiments  and  training  situations. 

3.  Testing  of  scenario  attributes  in  training  experiments  under  the  hypoth¬ 
esis  that  team  performance  can  be  predictably  improved  by  efficient  sce¬ 
nario  control  (see  Section  5.7.5). 

4.  Development  of  computer  capabilities  for  flexible  scenario  generation  ac¬ 
cording  to  the  team  training  and  experimentation  objectives  of  the  re¬ 
search  center  and  the  derived  principles  of  scenario  utility. 

Some  of  these  tasks  are  likely  to  form  a  part  of  any  research  using  simulation 
or  gaming  techniques.  But  they  also  represent  an  attempt  to  understand  the  rela¬ 
tionship  of  stimulus  material  to  resulting  learning.  A  machine-based  flexible  sce¬ 
nario  generator  would  reduce  training  costs  while  enhancing  training  capabilities. 
The  resource  requirements  for  the  research  are  similar  to  those  required  for  simu¬ 
lating  the  task  environment,  but  they  place  greater  emphasis  on  training  and 
experimental-design  specialists. 

5.3.3.  Capabilities  for  Distributed  Exercises8 

Decisionmaking  teams  are  typically  spatially  distributed.  While  the  spatial 
distribution  generates  requirements  for  communication,  cooperation,  and  coordi¬ 
nation  of  decisions  and  actions,  these  requirements  are  poorly  understood,  as  dem¬ 
onstrated  by  the  limitations  of  the  Naval  Tactical  Data  System  (NTDS),  currently 
the  best  system  supporting  distributed  decisionmaking.  The  distribution  of  team 
members  or  subteams  makes  it  generally  difficult  and  expensive  to  run  training 
exercises,  particularly  when  the  exercises  include  trainees  both  in  land-based 
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simulators  and  on  ships  at  sea.  Thus,  there  is  a  need  to  develop  training  capabilities 
that  can  support  distributed  exercises. 

Significant  technology-based  advances  toward  distributed  computing,  telecon¬ 
ferencing,  and  group  decisionmaking  have  recently  been  mediated  by  various  kinds 
of  communication  links  and  conferencing  protocols  (see  review  by  Scher,  1977). 
Research  sponsored  jointly  by  ARPA  and  the  Navy  through  the  Advanced  Com¬ 
mand  Architecture  Testbed  (ACCAT)  program  has,  for  example,  developed  graph¬ 
ics  teleconferencing,  satellite  message-handling  and  data-processing  systems — all 
connected  over  a  secure  packet-switched  network.  However,  only  a  few  distributed 
military  exercises  have  been  conducted,  and  these  have  not  exploited  or  en¬ 
couraged  the  technologies  required  for  improved  training  or  cooperative  task  per¬ 
formance. 

Plans  have  recently  been  developed  within  the  Navy  to  provide  some  of  the 
tools  required  for  distributed  exercises  and  training.  In  particular,  the  Naval  Ocean 
Systems  Center  in  San  Diego  is  developing  a  Warfare  Environmental  Simulator 
(WES)  running  on  a  Digital  Equipment  Corporation  (DEC)  VAX  11/780  to  be  placed 
aboard  all  carriers.  WES  provides  simulation  capabilities  for  naval  engagement 
exercises  and  can  be  expected  to  support  training  activities  on-board  each  carrier, 
if  not  between  two  or  more  carrier  task  forces. 

In  addition  to  a  standalone  simulator  and  training  system,  distributed  exercises 
require  facilities  to  coordinate  scenarios  involving  multiple  agents  (command-con¬ 
trol  or  force  elements)  on  multiple  computers.  This  requires  communications,  syn¬ 
chronization,  and  responsive  simulators. 

Two  major  approaches  to  distributed  exercises  seem  promising.  In  both  ap¬ 
proaches,  a  distributed  set  of  team  members  or  teams  participate  simultaneously. 
In  one  case,  all  of  the  participants  are  humans;  in  the  other  case,  some  of  the 
participants  are  replaced  by  computer-based  simulations.  The  primary  difference 
between  the  two  is  that  simulated  team  members  can  reduce  some  of  the  problems 
of  synchronization  and  feedback  that  arise  in  all-human  exercises. 

Existing  nets  and  teleconferencing  systems  should  provide  suitable  communica¬ 
tions  media  for  the  research.  However,  new  research  must  explore  cooperative 
problem-solving  techniques  using  these  media.  The  questions  to  be  addressed  in¬ 
clude  the  following:  How  can  distributed  problem-solvers  divide  and  conquer 
shared  tasks?  How  can  they  minimize  communications  and  response  times?  How 
can  they  manage  dynamically  changing  and  locally  dissimilar  situation  assess¬ 
ments?  New  software  aids  to  support  these  functions  will  be  required. 

Providing  simulated  training  problems  to  distributed  elements  will  create  band¬ 
width  and  synchronization  problems.  If,  for  example,  two  carrier  task  forces  are  to 
receive  sensor  reports  that  originate  simultaneously  from  one  simulated  engage¬ 
ment,  several  problems  arise.  These  will  probably  require  some  advances  in  simula¬ 
tion  technology  to  provide  a  basis  for  distributing  simulators  themselves,  so  that 
the  overall  scenario  can  be  controlled  from  one  processor  which  in  turn  synchro¬ 
nizes  two  local  subsimulators  at  distributed  sites.  The  local  subsimulators  would 
then  translate  the  high-level  commands  of  the  central  simulator  into  the  detailed 
events  that  trigger  sensor  reports  and  local  situation  displays.  This  architecture 
would  avoid  the  problem  of  excessive  data-communication  requirements  by  moving 
most  of  the  detailed  computing  to  local  processors.  Similar  multilevel  architectures 
had  to  be  developed  for  the  simpler  problem  of  network  graphics  communication 
in  the  ACCAT  program.  The  present  problem  will  require  research  on  the  best  way 
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to  create  multilevel  scenario  and  environment  simulators  that  support  distributed 
computing. 

This  effort  will  require  simulation  software,  networked  computers,  graphics, 
and  teleconferencing  software.  Cognitive  psychologists,  computer  scientists,  and 
military  personnel  familiar  with  the  distributed  exercise  will  be  needed  to  carry  out 
the  proposed  research. 


5.4.  DEVELOPMENT  OF  IMPROVED  PERFORMANCE 
MODELS  AND  EVALUATION  METRICS 

The  development  of  techniques  to  improve  team  performance  requires  an 
understanding  of  team  functions  and  techniques  for  measuring  team  behavior.  This 
section  describes  projects  aimed  at  the  modeling  and  evaluation  of  team  perfor¬ 
mance. 

5.4.1.  Cognitive  Models  of  Individual  Performance* 

A  first  step  in  modeling  the  performance  of  teams  must  be  the  detailed  analysis 
of  the  cognitive  processes  required  to  perform  the  tasks  of  individual  team  mem¬ 
bers,  e.g.,  enumeration  and  characterization  of  the  various  kinds  of  information 
received  by  an  air  intercept  controller  (AIC)  in  a  CIC.  A  performance  model  of  the 
AIC  would  specify  the  recognition,  classification,  transformation,  storage,  and  deci¬ 
sion  processes  he  uses  to  handle  enemy  air  threats.  This  cognitive  modeling  ap¬ 
proach  has  proven  effective  in  studies  of  individual  behavior  in  various  problem 
domains  (e.g.,  Bhaskar  and  Simon,  1977;  Brown  and  Burton,  1978;  Card,  Moran, 
and  Newell,  1980;  Collins,  Brown,  and  Larkin,  1977;  Greeno,  1978;  Hayes-Roth  and 
Hayes-Roth,  1979;  Newell  and  Simon,  1972;  Resnick,  1976).  Thus,  an  established 
research  technology  can  be  applied  to  this  aspect  of  team  performance. 

A  well-developed  cognitive  model  would  provide  a  basis  for  subsequent  re¬ 
search  on  the  training  and  composition  of  effective  teams.  Possible  applications 
include  (1)  selection  of  individuals  and  assignment  to  particular  team  roles  based 
on  their  performance  of  specific  information-processing  functions;  (2)  training  indi¬ 
viduals  in  the  component  information-processing  functions  they  will  be  required  to 
perform  (see  Section  5.7.1.);  (3)  diagnosis  of  faulty  information-processing  compo¬ 
nents  underlying  suboptima]  team  performance  (see  Sections  5.4.3,  5.5.1,  and  5.6.2); 
(4)  provision  of  informative  feedback  to  individual  team  members  (see  Section 
5.7.6);  and  (5)  designing  computer-based  aids  to  performance  (see  Sections  5  8. 1  and 
5.8.2). 

While  modeling  task  performance  of  team  members  can  utilize  many  of  the 
methods  of  earlier  studies,  it  will  also  require  an  important  conceptual  modifica¬ 
tion.  Because  the  earlier  studies  focused  exclusively  on  individual  performance, 
they  considered  only  information-processing  functions  performed  by  individuals  in 
isolation.  In  the  case  of  teams,  however,  interactions  among  team  members  consti¬ 
tute  a  large  part  of  the  task.  Thus,  comprehensive  task  analyses  must  also  describe 
information-processing  functions  that  mediate  interpersonal  communications. 
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Some  existing  analytic  techniques  (e.g.,  Bales,  1950)  may  be  adaptable  for  studying 
these  coordination  and  communication  functions. 

This  modeling  effort  is  a  low-risk  undertaking,  since  it  is  largely  a  new  appli¬ 
cation  of  existing  methodologies.  At  the  same  time,  it  is  extremely  important, 
because  the  results  will  provide  a  theoretical  basis  for  many  of  the  other  research 
issues. 

5.4.2.  Team  Members’  Mental  Models  for  Task  Performance10 

Many  cognitive  researchers  have  emphasized  the  importance  of  mental  models 
in  problem  solving  (Brown  and  Burton,  1975;  Stevens  and  Collins,  1978;  Chi  and 
Glaser,  1979;  Hayes-Roth,  1980;  Larkin,  1979;  Schank  and  Abelson,  1977;  Simon 
and  Simon,  1978).  A  mental  model  is  the  problem-solver’s  internal  representation 
of  the  problem  at  hand.  It  comprises  explicit  or  implicit  assumptions  regarding  the 
various  objects  in  the  problem,  the  behaviors  of  these  objects,  and  the  interactions 
among  them.  Mental  models  frequently  instantiate  these  objects  and  behaviors  by 
analogy  to  some  known  physical  or  symbolic  system.  (For  example,  one  may  under¬ 
stand  electricity  by  analogy  to  a  hydraulic  flow  model.)  The  model  may  also  entail 
assumptions  about  the  details  of  the  goal  and  about  knowledge  and  strategies  that 
would  be  useful  in  achieving  the  goal.  Mental  models  are  important  because  they 
determine  the  problem-solver’s  approach  to  the  problem.  This,  in  turn,  determines 
the  ease  with  which  he  finds  a  solution  and  the  quality  of  the  solution. 

The  concept  of  mental  models  relates  to  team  performance  in  two  ways.  First, 
mental  models  influence  the  problem-solving  performance  of  individual  team  mem¬ 
bers.  Team  members  may  perform  better  if  they  understand  how  their  task  relates 
to  the  other  members  with  whom  they  interact  as  well  as  to  overall  mission  goals. 
Second,  the  relationships  among  mental  models  held  by  various  team  members 
may  influence  their  performance  as  a  team.  For  example,  if  team  members  share 
the  same  mental  model,  this  might  facilitate  overall  decisionmaking — they  would 
work  under  the  same  assumptions,  would  have  shared  expectations  about  the 
responsibilities  and  procedures  of  various  members,  and  would  use  similar  decision 
strategies.  Their  shared  conceptualization  should  facilitate  communication,  coordi¬ 
nation  of  decisionmaking  activities,  and  cooperation  in  solving  particular  subprob¬ 
lems.  On  the  other  hand,  sharing  the  same  mental  model  might  inhibit  team  perfor¬ 
mance  if  team  members  became  "locked”  into  an  unproductive  approach  to  a 
problem. 

It  is  also  possible  that  having  different  mental  models  might  facilitate  other 
aspects  of  performance.  For  example,  by  trying  alternative  decision  strategies 
entailed  by  the  different  mental  models  of  various  team  members,  the  team  might 
benefit  from  synergistic  interactions  between  the  products  of  two  or  more  ap¬ 
proaches.  Effective  exploitation  of  alternative  mental  models  probably  would  re¬ 
quire  an  awareness  of  the  characteristics  of  different  models  and  intelligent  deci¬ 
sions  about  how  to  coordinate  them. 

Given  this  analysis,  new  research  should  address  the  following  questions: 

1.  How  can  the  mental  models  of  team  members  on  information-processing 
decisionmaking  teams  be  represented?  One  approach,  suggested  by  Col- 
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lins  (1980),  would  model  the  goal  hierarchies  of  various  team  members. 
Alternatively,  models  might  reflect  emphases  on  procedures  or  informa¬ 
tion  flow  rather  than  on  goals. 

2.  How  do  team  members’  mental  models  differ? 

3.  What  are  the  advantages  and  disadvantages  of  specific  mental  models? 

4.  What  are  the  performance  consequences  of  team  members  using  the  same 
or  different  mental  models? 

5.  How  do  particular  combinations  of  models  interact? 

The  proposed  work  would  constitute  basic  research  in  a  relatively  unexplored 
area.  As  such,  it  must  be  considered  somewhat  risky.  If  successful,  however,  the 
research  would  have  a  high  payoff,  since  decisionmaking  teams  currently  work 
with  ill-structured  problems,  poorly  specified  problem-solving  methods,  and  loosely 
defined  SOPs. 

5.4.3.  Investigation  of  the  Relationship  between  Individual  and 
Team  Performance" 

The  necessity  for  a  research  program  on  team  training,  in  addition  to  individual 
training,  rests  on  the  assumption  that  team  behavior  represents  more  than  aggre¬ 
gated  individual  behavior.  Informal  simulation  experiments  have  demonstrated 
that  team  decisionmaking  performance  can  continue  to  change  after  individual 
member  performance  has  stabilized  (Weiner,  1960).  However,  the  relationship  be¬ 
tween  individual  and  team  performance  must  be  systematically  investigated  to 
determine  the  appropriateness  and  utility  of  team  studies. 

New  research  in  this  area  should  address  three  related  issues:  (1)  the  relation¬ 
ship  between  a  team’s  effectiveness  and  that  of  its  members,  (2)  the  influence  of  the 
leader’s  skills  and  behavior  on  overall  performance,  and  (3)  the  extent  to  which 
training  should  focus  on  entire  teams  (i.e.,  communication  and  coordination  skills) 
or  on  individual  members  (i.e.,  individual  operator  skills). 

Two  observations  should  guide  the  design  of  experimental  studies  in  this  area. 
First,  there  is  not  a  strict  dichotomy  between  team  and  individual  training;  and 
second,  observations  of  individual  and  team  performance  should  be  made  in  the 
team’s  working  environment  or  a  simulation  of  that  environment.  Whether  it  does 
so  implicitly  or  explicitly,  individual  training  for  later  team  membership  must 
assume  a  task  structure  and,  more  generally,  the  team’s  working  environment. 
Effective  transfer  of  individual  training  may  depend  on  accurate  simulation  of  the 
team  environment.  Moreover,  team  and  individual  training  may  be  combined. 
Individual  training  of  basic,  elemental  skills  may  precede  team  training  of  com 
munication  skills  and  behavior  in  unusual  situations,  which  often  takes  place  in 
shipboard  exercises. 

Four  aspects  of  the  research  in  this  area  deserve  special  attention: 

1.  Measurement  of  Individual  and  Team  Performance.  Two  factors  interfere 
with  the  straightforward  assessment  of  the  performance  of  information-processing 
decisionmaking  teams.  First,  "true”  performance  occurs  only  during  actual  battle, 
and  thus  little  data  are  available  from  which  to  assess  effectiveness.  Second,  team 
performance  may  be  assessed  either  for  its  efficiency  or  for  its  ffectiveness.  Efficiency 
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refers  to  easily  obtained  measurements  of  the  process  characteristics  of  team  per¬ 
formance,  including  detection  errors,  plotting  errors,  response  time  to  threats,  and 
accuracy  of  weapons  targeting.  Effectiveness  refers  to  the  ability  of  the  team  to 
achieve  the  mission  objectives,  which  typically  require  the  team  to  defuse  the 
enemy  threat  with  minimal  loss  to  its  own  resources  and  within  the  constraints 
specified  by  the  battle  group  commander.  Thus,  performance  of  both  individuals 
and  teams  can  be  measured  in  a  variety  of  ways  and  at  different  levels  of  abstrac¬ 
tion.  While  team  efficiency  bears  some  relationship  to  effectiveness,  this  relation¬ 
ship  is  in  practice  simply  assumed,  but  in  fact  it  is  rarely  known.  Research  will  be 
required  to  understand  this  relationship,  to  devise  techniques  for  aggregating  indi¬ 
vidual  performance,  and  for  directly  contrasting  it  with  measures  of  team  perfor¬ 
mance. 

Currently  available  measures  of  performance  in  operational  training  are  based 
on  subjective  ratings  by  instructors.  Such  ratings,  however,  are  prone  to  various 
types  of  systematic  and  random  error  (Landy  and  Farr,  1980).  Thus,  new,  objective 
performance  measures  must  be  developed  based  on  analyses  of  the  task  require¬ 
ments  (see  Section  5.4.1). 

2.  Determination  of  Individual  Contributions  to  Team  Performance.  Re¬ 
search  must  improve  methods  for  determining  how  an  individual’s  actions  affect 
overall  outcomes  in  team  performance,  so  that  the  criticality  of  the  various  individ¬ 
ual  roles  can  be  assessed.  One  approach  to  this  problem  would  be  to  interpret 
overall  outcomes  in  terms  of  causal  precursor  events  and  then  attribute  responsibil¬ 
ity  to  appropriate  actors.  To  assist  in  this  analysis,  some  evaluation  should  be  made 
of  alternative  courses  of  action  that  were  open  and  what  their  effects  would  have 
been. 

3.  Analysis  of  Task  Independence,  Concreteness,  and  Segmentation.  The  rela¬ 
tionship  between  the  performance  of  team  members  and  the  overall  performance 
of  the  team  may  well  depend  on  attributes  of  the  task.  Concrete  (e.g.,  requiring 
motor  rather  than  cognitive  skills)  or  segmented  (i.e.,  separated  in  time  or  space) 
team  performance  may  be  simply  related  to  individual  performances.  In  this  case, 
individual  training  should  be  sufficient  to  produce  good  performance.  However, 
these  cass  fail  to  obtain  when  individual  tasks  are  complex  or  ill-defined,  when 
coordination  among  team  members’  activities  is  essential,  or  when  the  organization 
of  tasks  and  communications  within  the  team  is  flexible.  In  such  cases,  team  perfor¬ 
mance  depends  heavily  on  intrateam  interaction  and  leadership,  and  team  training 
should  especially  improve  performance. 

4.  Analysis  of  Environmental  Predictability  and  Control.  The  relative  impor¬ 
tance  of  individual  and  team  skills  may  also  be  governed  by  the  predictability  of 
the  operating  environment.  The  more  the  training  environment  (including  enemy 
weapons  and  actions,  the  reliability  of  equipment  and  resources,  and  the  behaviors 
of  other  team  members)  resembles  the  conditions  experienced  during  training 
exercises,  the  more  team  performance  may  simply  relate  to  individual  perfor¬ 
mances.  However,  in  unpredictable  environments  (and  especially  in  combat  situa¬ 
tions  that  cannot  be  handled  successfully  by  the  team’s  weapons  and  SOPs),  team 
performance  may  depend  more  heavily  on  team  coordination  skills  and  effective 
leadership.  In  such  situations,  teams  cannot  successfully  follow  SOPs.  Team  mem¬ 
bers  must  not  only  change  behaviors,  they  must  coordinate  the  new  behaviors. 
Effective  leadership,  experience  as  a  team,  and  good  working  relations  among  team 
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members  (perhaps  brought  about  by  team  training)  should  speed  adaptation  to  the 
new  environment  and  therefore  improve  performance. 

This  research  is  central  to  the  interests  of  a  Team  Performance  Research 
Center,  since  it  focuses  on  the  determination  of  team  training  requirements.  The 
proposed  studies  will  require  the  use  of  a  team  performance  laboratory  to  provide 
training,  to  provide  problems  on  which  performance  can  be  measured,  and  to 
record  performance  measures.  Expertise  will  be  required  in  measurement,  experi¬ 
mental  methodology,  instruction,  and  military  simulation  and  gaming. 

5.4.4.  Theories  of  Team  Performance12 

Simply  stated,  there  is  no  adequate  theory  of  team  performance.  Such  a  theory 
should  describe  the  major  factors  affecting  team  performance  and  should  enable 
interpretation  or  prediction  of  the  effects  of  interventions  in  the  team  training  and 
performance  process. 

Many  variables  and  processes  interact  in  determining  the  performance  of  a 
team.  These  include  individual  and  team  morale,  retention  rates,  individual  skills, 
individual  and  team  incentives,  individual  and  team  objectives,  experience  levels, 
homogeneity  or  variability  of  personnel  backgrounds,  and  task  difficulty.  The  de¬ 
velopment  of  a  theoretical  framework  requires  formulating  the  key  concepts, 
hypothesizing  the  causal  relations  among  them,  and  empirically  testing  predicted 
relationships.  With  a  theory  in  hand,  one  should  be  able  to  identify  the  crucial 
factors  for  Navy  intervention.  Without  such  a  theory,  as  is  currently  the  case,  one 
can  only  guess  about  the  relative  impact  of  alternative  programmatic  plans. 

Prior  research  on  teams  in  other  domains  may  provide  a  source  of  initial  ideas 
for  this  research.  Studies  of  sports  teams  and  industrial  groups  have  produced 
informal  theories  of  the  influence  of  training,  feedback  on  performance,  motivation, 
etc.  Much  previous  experimental  research  on  team  performance  has  abstracted 
tasks  from  real  operating  environments  to  observe  how  theoretically  derived  vari¬ 
ables  influence  team  behavior  (e.g.,  Steiner,  1972;  Ofshe,  1973;  Meister,  1976). 
Studies  of  this  type  typically  manipulate  variables  such  as  stimulus  inputs  and 
reinforcement  schedules,  performance  sequencing,  team  size,  and  other  factors  that 
derive  from  theories  of  learning  or  group  behavior.  However,  how  the  experimen¬ 
tal  measures  of  performance  relate  to  actual  team  performance  and  team  effective¬ 
ness  is  not  known. 

Research  aimed  at  developing  a  theory  of  team  performance  should  concen¬ 
trate  on  two  areas: 

1.  Development  of  an  Aggregate  Theory  of  Team  Performance.  Such  a  theory 
might  take  the  form  of  a  causal  network  expressing  dependencies  among  social, 
organization,  and  cognitive  variables.  A  similar  approach  has  been  adopted  in  work 
on  job  design  for  individuals  (Hackman,  Oldham,  Janson,  and  Purdy,  1975;  Barrett, 
Alexander,  and  Dambrot,  1979).  A  causal  network  for  teams  might,  for  example, 
relate  esprit  de  corps  and  skill  level  to  performance  under  stress,  performance 
under  varying  stress  conditions  to  job  satisfaction,  satisfaction  and  promotion  poli¬ 
cies  to  longevity  on  the  team,  and,  recursively,  longevity  to  esprit  de  corps.  The 
initial  framework  should  reflect  the  significant,  plausible  influences  on  team  perfor- 
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mance.  These  might  include,  in  addition  to  those  cited  above,  person-  and  task- 
specific  indices  such  as  measures  of  the  complexity,  difficulty,  and  variability  of  job 
roles;  personnel  intelligence;  on-the-job  experience;  and  professional  commitment. 
These  initial  hypothetical  relationships  would  motivate  empirical  or  experimental 
assessments,  perhaps  using  covariate  structures  analysis  or  longitudinal  experi¬ 
mental  methods  (see  Hunt,  1980). 

2.  Development  of  a  Micro-Theory  of  Team  Performance.  A  micro-theory 
would  elaborate  team  performance  phenomena  within  the  problem-solving  frame¬ 
work  of  cognitive  science.  Such  an  analysis  of  team  behavior  would  articulate  the 
team’s  goals,  its  information-processing  resources,  the  hypotheses  it  must  generate 
and  evaluate  in  the  course  of  working  toward  its  goals,  its  overall  organization,  and 
its  control  procedures  (see  Section  5.4.1).  In  addition,  this  viewpoint  aims  at  build¬ 
ing  a  theory  in  finer  detail  than  an  aggregate  approach  would  permit. 

Two  particular  problem-solving  models  seem  promising  for  this  approach.  This 
first  would  model  the  team  as  a  causal  network  with  input-output  dependencies 
between  team  members  (e.g.,  between  the  producers  of  intermediate  decisions  and 
the  subsequent  consumers).  The  second  would  view  a  team  as  a  loosely  coupled 
collection  of  problem-solving  specialists  (see  Section  5.6.2).  This  model  raises  ques¬ 
tions  about  how  decisions  are  communicated  between  specialists,  how  one’s  actions 
affect  others’  concurrent  or  subsequent  responses,  and  the  overall  coordination  of 
problem-solving  efforts. 

Given  an  initial  structural  framework,  human  or  machine  experiments  could 
test  hypotheses  about  performance  determinants.  Specifically,  particular  command 
structures,  communications  networks  (Section  5.6.3),  types  of  decisionmaking  col¬ 
laboration  (Section  5.6.2),  or  task  definitions  (Section  5.6.1)  of  theoretical  interest 
could  be  empirically  tested  by  evaluating  the  performance  of  an  experimental  team 
having  the  appropriate  characteristics.  The  team  could  consist  of  several  humans 
following  prescribed  rules  of  behavior  or,  in  a  completely  controlled  test,  the  team 
could  be  composed  of  artificial  (computer-based)  team  members  following  the  same 
rules.  Both  types  of  experiments  have  been  conducted  previously  in  several  task 
domains  (e.g.,  Hayes-Roth  and  Wesson,  1978;  CMU  Computer  Science  Research 
Group,  1977).  Another  hypotheses-testing  method  might  contrast  the  processes  and 
performance  of  expert  teams  with  those  of  novice  teams.  Such  comparisons  could 
be  made  either  for  real  Navy  teams  or  for  teams  formed  and  trained  in  the  labora¬ 
tory. 

Such  research  would  require  expertise  in  military  decisionmaking,  manpower, 
training,  social  and  cognitive  psychology,  and  computer  science,  especially  the  area 
of  cooperative  problem-solving.  Long-term  studies  of  manpower  and  personnel  data 
would  require  access  to  existing  data  bases  and/or  new  longitudinal  studies.  Both 
human  and  machine  experimental  studies  should  be  conducted  using  interactive 
computer-based  simulation  and  communication  facilities.  Computer-based  experi¬ 
mental  studies  could  be  performed  best  with  a  network  of  powerful  midsize  comput¬ 
ers  (e.g.,  Digital  VAX  11/780,  Xerox  DORADO,  or  perhaps  Three-Rivers  PERQ) 
supporting  artificial  intelligence  programming  languages  (e.g.,  LISP). 

This  research  should  be  considered  somewhat  long-range,  although  early  re¬ 
sults  might  be  obtained  in  the  area  of  aggregate  modeling  from  existing  manpower 
data.  The  aggregate  effort  should,  within  a  few  years,  influence  policy  decisions 
regarding  team  performance  questions.  For  example,  alternative  rotation  schemes 
might  be  found  that  simultaneously  increased  assignment  durations,  esprit  de 
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corps,  retention,  and  individual  and  team  performance.  The  micro-theoretical  effort 
could,  if  successful,  have  a  tremendous  impact  on  management  science,  organiza¬ 
tional  design,  cognitive  science,  systems  engineering,  and  computer  science.  This 
impact  would  result  from  the  ability  to  apply  the  performance  theory  to  practical 
questions  for  which  no  engineering  practices  or  guidelines  exist. 


5.5.  INVESTIGATION  OF  TEAM  SYNERGY  AND  TURNOVER 

It  is  axiomatic  that  team  performance  changes  over  time  and  with  practice. 
However,  the  development  of  cohesive  and  efficient  teamwork  is  frequently  dis¬ 
rupted  by  personnel  turnover.  One  approach  to  improving  team  performance  in¬ 
volves  compensation  for  the  disruptive  effects  of  turnover.  The  following  studies 
based  on  this  approach  should  be  included  in  the  team  performance  research  pro¬ 
gram. 


5.5.1.  The  Dynamics  of  Team  Evolution13 

When  a  team  first  assembled,  the  members  perform  the  activities  of  the  team 
with  a  great  deal  of  confusion  and  with  rather  poor  results.  With  continued  prac¬ 
tice,  however,  the  team  may  perform  in  a  highly  skilled  manner.  Little  research  has 
systematically  investigated  the  dynamics  of  team  performance  evolution.  Section 
5.4.3  posits  circumstances  under  which  a  team’s  performance  is  not  simply  predict¬ 
able  from  the  performance  of  individual  team  members.  If  this  hypothesis  is  cor¬ 
rect,  the  issue  of  how  a  collection  of  individuals  evolves  into  an  efficient  organiza¬ 
tion  is  central  to  a  team  performance  research  program. 

The  concept  of  team  learning,  as  opposed  to  an  aggregation  of  the  learning  of 
individual  team  members,  has  received  only  limited  research  attention.  The  Sys¬ 
tems  Research  Laboratory  at  The  Rand  Corporation,  the  Naval  Research  Labora¬ 
tory,  and  a  few  other  facilities  pioneered  interest  in  man-machine  systems  research, 
particularly  for  information-processing  teams  (Parsons,  1972).  The  Rand  research 
efforts  provided  some  of  the  initial  conceptual  formulations  of  team  evolution  or 
"system  growth”  (Weiner,  1960).  This  research  identified  the  importance  of  having 
explicit  team  goals,  of  providing  feedback  on  the  results  of  team  performance,  and 
of  allowing  teams  to  change  procedures,  develop  a  special  language,  and  alter  rules 
to  cope  with  increasingly  difficult  tasks.  It  also  provided  a  framework  for  viewing 
team  evolution  as  an  interaction  among  three  "environments”:  the  task,  the  equip¬ 
ment,  and  the  sociocultural  environment  of  the  team. 

This  framework  provides  a  rich  source  of  questions  that  merit  further  research: 

1.  What  are  the  stages  or  phases  through  which  a  decisionmaking  team 
evolves?  Do  teams  always  progress  through  the  same  stages?  How  do 
these  stages  correspond  to  the  development  of  individual  member  skills? 

2.  What  mechanisms  or  processes  in  the  task,  equipment,  or  sociocultural 
environments  stimulate  or  underlie  change? 

3.  How  do  teams  modify  their  standard  operating  procedures  (SOPs)  and 
their  decisionmaking  processes  over  time  and  with  practice?  How  do  they 
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develop  their  own  "shared  language”?  Do  these  modifications  occur  con¬ 
tinuously  or  only  at  certain  stages  of  evolution?  Do  team-specific  SOPs  and 
shared  languages  in  fact  increase  team  efficiency  and  ease  intra-team 
communication?  Do  such  team  idiosyncrasies  cause  difficulties  in  com¬ 
municating  outside  the  team?  Does  the  evolution  of  team-specific  SOPs 
belie  outsiders’  understanding  of  team  behavior? 

4.  How  can  these  mechanisms  or  processes  be  incorporated  into  team  train¬ 
ing  technologies  or  strategies?  Are  team  members  aware  of  and  can  they 
learn  to  control  evolutionary  changes? 

Early  studies  of  team  evolution  provided  only  limited  data  on  these  issues,  most 
of  which  were  derived  from  elaborate  simulations  of  military  installations  such  as 
Navy  CICs  and  Air  Force  Air  Defense  Direction  Centers.  Simulation  still  appears 
to  be  an  appropriate  methodology  for  the  research,  since  an  important  factor  in 
team  performance  is  the  realistic  portrayal  of  consequences  of  team  actions.  Thus, 
a  simulation  laboratory  and  the  cooperative  efforts  of  specialists  in  military  gam¬ 
ing,  computer  simulation,  graphics,  experimental  methodology,  and  protocol  analy¬ 
sis  will  be  required.  If  successful,  this  research  can  not  only  influence  team  perfor¬ 
mance,  it  can  also  lead  to  system  and  equipment  designs  that  enhance  overall 
operational  effectiveness. 

5.5.2.  The  Effects  of  Team  Turnover  on  Performance14 

The  evolution  of  team  performance  is  frequently  disrupted  by  personnel  turn¬ 
over.  While  previous  studies  have  investigated  some  of  the  factors  influencing 
turnover  (Porter  and  Steers,  1973)  and  the  impact  of  turnover  on  system  design 
(Meister,  1976),  little  or  no  research  has  explored  the  impact  of  turnover  on  team 
performance.  Section  5.1.4  outlines  proposed  policy  studies  of  the  impact  of  turn¬ 
over  on  Navy  teams.  In  addition,  new  research  should  assess,  in  the  experimental 
laboratory,  the  disruptive  effects  of  turnover  and  the  efficacy  of  various  techniques 
for  reducing  this  disruption. 

The  high  degree  of  turnover  in  naval  teams  must  result  in  considerable  cost  to 
overall  performance.  Team  members  must  frequently  learn  new  jobs  and  must 
learn  to  interact  effectively  with  new  teammates.  Rapidly  changing  teams  are 
unable  to  develop  "empathetic  models,”  or  models  of  the  goals,  skills,  and  tasks  of 
other  team  members  (see  Section  5.7.3).  Moreover,  they  may  not  have  time  to 
evolve  specialized  interaction  protocols  or  operating  procedures,  as  described  in 
Section  5.5.1.  In  short,  excessive  turnover  may  prevent  teams  from  performing 
beyond  the  level  predicted  by  the  simple  aggregation  of  individual  behaviors.  On 
the  other  hand,  members  of  completely  static  teams  may  become  overly  dependent 
upon  their  particular  teammates  and  the  performance  protocols  and  mental  models 
they  have  developed  together  (see  Section  5.4.2).  When  unanticipated  situations 
arise,  highly  structured  and  integrated  teams  may  be  too  inflexible. 

One  major  thrust  in  this  research  area  should  be  an  exploration  of  the  effects 
of  replacing  one  or  more  team  members  during  various  stages  of  the  team’s  evolu¬ 
tion.  Are  the  old  members  of  the  team  able  to  compensate  rapidly  for  the  introduc¬ 
tion  of  new  members?  A  second  focal  area  for  research  would  be  an  investigation 
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of  techniques  for  compensating  for  the  disruptive  effects  of  turnover.  For  example, 
members  of  stable  teams  might  benefit  from  different  types  of  training  than  are 
effective  for  turbulent  teams.  Members  of  stable  teams  might  benefit  most  from 
training  with  their  own  team  members  (developing  empathetic  models,  interaction 
protocols,  and  so  on),  while  members  of  turbulent  teams  might  benefit  from  training 
with  a  variety  of  other  team  personnel,  where  they  would  be  given  experience  in 
cooperating  well  with  a  variety  of  individuals.  These  people  might  also  benefit  from 
specific  training  in  social  and  functional  adaptability. 

Such  studies  of  the  costs  and  benefits  of  turnover  require  relatively  long-term 
observation  of  team  performance  in  the  laboratory.  Experimental  studies  manipu¬ 
lating  two  or  three  levels  of  turnover  would  probably  be  sufficient  to  observe 
significant  effects.  The  research,  if  successful,  will  have  two  payoffs:  (1)  it  will 
provide  knowledge  of  the  consequences  of  existing  team  composition  policies,  and 
(2)  in  conjunction  with  policy  decisions  about  acceptable  levels  of  personnel  turn¬ 
over,  it  will  constrain  the  set  of  approaches  that  are  likely  to  improve  performance. 


5.6.  IMPROVEMENT  OF  TEAM  ORGANIZATION 

Most  information-processing  decisionmaking  teams  have  relatively  loose  and 
flexible  organizations.  Thus,  team  performance  might  be  improved  through  the 
determination  of  effective  organization,  communication,  and  decisionmaking  struc¬ 
tures. 

Several  such  studies  of  team  organization  are  described  below. 

5.6.1.  Task  Restructuring15 

Complex  information-processing  systems  are  increasing  rapidly  both  in  number 
and  in  importance  for  military  systems.  A  basic  supposition  is  that  information 
systems  increase  weapon  effectiveness  through  increased  data  acquisition  and 
precision.  However,  such  complex  systems  also  increase  the  processing  demands  on 
teams  and  the  requirements  for  combining  and  evaluating  knowledge.  Thus,  it  is 
necessary  to  develop  a  methodology  for  determining  task  structures  that  will  en¬ 
hance  the  use  and  performance  of  these  systems. 

The  allocation  of  operating  tasks  to  team  members  in  military  systems  is  usu¬ 
ally  determined  by  system  design  and  by  operating  procedures  developed  as  part 
of  the  design  process.  Tasks  are  often  allocated  according  to  Service  policies  and 
organizational  factors.  For  example,  the  most  important  decisionmaking  in  CICs  is 
performed  by  members  with  the  highest  ranks.  Allocation  of  tasks  in  terms  of 
system  design  and  institutional  constraints  may  not  lead  to  the  most  efficient  utili¬ 
zation  of  team  resources.  Furthermore,  the  allocation  of  individuals  to  tasks  within 
a  team  varies  from  ship  to  ship  according  to  the  desires  of  the  commanding  officer. 
Hence,  research  should  consider  methods  for  task  structuring  that  could  improve 
team  performance. 

The  most  hasic  issue  regarding  task  allocation  is  the  determination  of  the  joint 
requirements  for  task  performance  and  for  information  at  various  system  levels. 
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Task  allocations  are  often  based  on  broad  policies  that  do  not  take  into  account 
their  effect  on  system  and  team  performance.  For  example,  the  policy  to  centralize 
command  decisions  in  CICs  results  in  most  decisions  being  made  at  high  levels,  the 
flooding  of  information  channels,  the  requirement  for  rapid  and  accurate  aggrega¬ 
tion  of  information,  and  delays  in  response  time.  Thus,  even  when  low-level  sensor 
data  have  high  fidelity,  system  response  may  be  poor  as  a  result  of  failure  to  rapidly 
aggregate  the  data,  assess  the  situation,  make  decisions,  and  execute  responses. 
Research  is  needed  to  determine  how  to  structure  tasks  requiring  decisionmaking, 
plan  execution,  and  information  transmission  to  optimize  team  effectiveness.  The 
success  of  previous  attempts  to  influence  team  performance  through  manipulation 
of  the  task  structure  (e.g.,  Weiner,  1964;  Lawrence  and  Lorsch,  1967;  Shaw,  1971; 
Hayes-Roth  and  Wesson,  1978)  indicates  the  promise  of  this  approach. 

Another  issue  is  the  influence  of  key  personnel  on  decisionmaking  teams  with¬ 
out  rigid  procedures  and  strict  task  allocations.  A  recent  Air  Force  assessment  of 
operating  Tactical  Command  and  Control  Systems  found  that  actual  task  struc¬ 
tures  were  often  derived  from  the  influence  of  a  single  resourceful  and  energetic 
person  who  controlled  the  system  (or  a  large  part  of  it),  rather  than  from  standard¬ 
ized  procedures.  Teams  that  depend  heavily  on  particular  individuals  may  degrade 
dramatically  when  these  key  people  are  lost  or  overloaded.  New  research  would 
develop  both  methods  for  allocating  tasks  that  exploit  such  resources  effectively 
and  methods  for  reallocating  resources  when  key  personnel  are  removed. 

This  research  will  require  the  use  of  laboratory  simulations  and  experiments. 
While  it  could  have  some  relatively  short-range  payoffs,  it  is  most  promising  as  a 
rather  long  range  (over  two  years),  sequential  effort  in  which  earlier  studies  pro¬ 
vide  design  requirements  for  later  ones. 

5.6.2.  Investigation  of  Alternative  Decisionmaking  Strategies'6 

In  the  past  few  years,  cognitive  scientists  have  developed  team-like  models  of 
individual  cognitive  processing  (Newell  and  Simon,  1972;  Lenat,  1975;  CMU  Com¬ 
puter  Science  Research  Group,  1977;  Engelmore  and  Nii,  1977;  Hayes-Roth  and 
Hayes-Roth,  1979;  Waterman  and  Hayes-Roth,  1978).  These  models  assume  that 
individuals  possess  many  functionally  independent  cognitive  "specialists,”  each  of 
which  has  its  own  expertise  which  it  can  bring  to  bear  on  particular  problems. 
Ordinarily,  problem  solution  requires  the  combined  performance  of  many  such 
specialists. 

The  power  of  these  models  lies  not  only  in  the  expertise  of  the  individual 
specialists,  but  also  in  the  "executive”  strategies  that  organize  their  activities.  For 
example,  in  their  work  on  the  Hearsay  II  speech-understanding  system,  the  CMU 
group  experimented  with  several  different  executive  strategies.  Eventually  they 
decided  that  the  optimal  executive  strategy  for  their  task  was  to  perform  detailed 
analyses  of  the  speech  signal  before  generating  hypotheses  about  higher-level, 
more  abstract,  syntactic  or  semantic  interpretations.  In  studies  of  planning  behav¬ 
ior,  Hayes-Roth  (1980)  found  that  similar  "bottom-up”  strategies  worked  well  for 
some  problems,  while  "top-down"  strategies  worked  better  for  others.  Thus,  indi¬ 
vidual  performance  appears  to  benefit  from  different  strategies,  depending  upon 
the  task  and  upon  the  characteristics  of  the  specific  problem  at  hand. 
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These  team-like  models  of  individual  behavior  provide  a  promising  framework 
for  studying  the  behavior  of  decisionmaking  teams.  The  cognitive  specialists  corre¬ 
spond  to  the  individual  members  of  a  team  and  the  executive  strategies  correspond 
to  the  organization  of  the  team’s  activities.  Thus,  new  research  might  investigate 
the  effects  of  executive  strategies  on  team  performance. 

Extrapolating  from  the  work  on  individual  problem-solvers,  one  might  expect 
alternative  executive  strategies  to  work  well  for  different  classes  of  problems  and 
for  specific  problems  within  a  class.  For  example,  a  top-down,  hierarchical  executive 
structure  ought  to  work  well  for  teams  that  deal  with  a  single,  well-structured 
problem,  while  a  more  decentralized,  opportunistic,  bottom-up  approach  should 
work  well  for  teams  that  deal  with  ill-structured  problems  having  many  alternative 
actions  (March,  1972;  Hayes-Roth  and  Hayes-Roth,  1979;  Hunt,  1980). 

Given  this  interaction  between  problem  and  strategy,  it  may  be  necessary  for 
teams  to  self-organize,  adopting  alternative  executive  strategies  as  they  evolve  (see 
Section  5.5.1).  Reorganization  might  entail  changes  in  individual  roles  and  respon¬ 
sibilities  as  well  as  changes  in  the  way  individuals’  activities  are  coordinated. 
Effective  self-organization  requires  that  one  or  more  team  members  understand  the 
relationships  between  problem  characteristics  and  team  organization.  It  also  re¬ 
quires  that  team  members  be  flexible  enough  to  assume  different  roles  and  func¬ 
tions  under  different  circumstances.  This  in  turn  may  require  that  individual  team 
members  have  mental  models  of  task  demands  and  expertise  extending  well  beyond 
their  ordinary  responsibilities  (see  Section  5.4.2). 

Another  important  aspect  of  team  organization  is  the  locus  of  decisionmaking 
responsibility.  Prior  research  in  social  psychology  indicates  that  decisions  reached 
through  group  deliberation  are  more  risky  than  those  made  by  individuals  (Ofshe, 
1973).  While  some  research  has  investigated  the  processes  of  group  decisionmaking 
on  real  strategic  defense  problems  (Janis,  1977),  the  case  study  methodology  used 
in  the  research  prevents  the  unconfounded  comparison  of  alternative  decision 
methods. 

Given  this  analysis,  new  research  should  address  the  following  questions  con¬ 
cerning  the  procedures  for  organizing  team  decisionmaking: 

1.  What  alternative  executive  strategies  might  be  applied  to  decisionmaking 
tasks? 

2.  Are  people  equally  adept  at  conforming  to  these  strategies? 

3.  What  impact  does  choice  of  strategy  have  on  task  performance? 

4.  Overall,  which  strategies  are  optimal  for  which  classes  of  problems? 

5.  Can  teams  learn  alternative  executive  strategies  and  adopt  them  appro¬ 
priately  for  particular  problems? 

6.  What  characteristics  identify  individuals  who  can  operate  flexibly  under 
alternative  strategies? 

7.  What  side-effect3  does  training  in  flexibility  produce  (e.g.,  higher  job  satis¬ 
faction,  better  personal  relations)? 

8.  Overall,  what  is  the  cost/benefit  tradeoff  between  training  flexible  versus 
"fixed-configuration”  teams? 

These  studies  could  easily  be  conducted  in  a  simulated  task  environment  by 
cognitive  and  social  psychologists.  The  research  has  high  potential  payoff  in  terms 
of  team  efficiency  and  ability  to  operate  under  changing  conditions.  It  could  easily 


be  conducted  on  a  low-risk  schedule,  with  the  major  body  of  research  contingent 
upon  promising  outcomes  for  Questions  1  and  3. 


5.6.3.  Investigation  of  Alternative  Communications  Structures” 

As  pointed  out  in  Section  5.6.1,  communication  is  a  critical  problem  in  time- 
stressed  team  decisionmaking.  Messages  among  cooperating  decisionmakers  vary 
in  their  time  criticality,  the  reactions  they  demand,  and  the  follow-ups  they  entail. 
Messages  about  most  current  and  future  events  require  time-indexing.  Maintaining 
a  coherent  picture  of  the  world  at  any  point  in  time  requires  the  combination  and 
evaluation  of  time-synchronized  messages.  A  crucial  problem  in  team  performance 
is  that  of  identifying  the  kinds  of  information  that  must  be  communicated  and  then 
increasing  the  system’s  efficiency  at  handling  that  type  of  communication. 

Most  work  on  communications  has  been  conducted  in  isolation.  Advances  in 
hardware  have  focused  on  bandwidths  and  protocols;  advances  in  software  have 
focused  on  message  systems  and  teleconferencing;  research  on  group  problem¬ 
solving  has  measured  the  quantity  of  various  types  of  speech  acts  within  various 
group  communication  structures;  work  on  natural  language  has  focused  primarily 
on  the  meaning  of  messages  between  participants.  Communications  research  in  the 
domain  of  team  performance  must  relate  the  nature  of  the  decision  task  to  the 
function  of  communication  within  that  task  environment.  Therefore,  the  overall 
task  must  be  viewed  as  an  organization  of  subtasks;  communication  is  the  transpor¬ 
tation  system  of  such  an  organization,  moving  products  from  producers  to  consum¬ 
ers.  The  adequacy  of  a  communication  system  depends  on  its  capacity  for  handling 
expected  volumes,  the  efficiency  of  its  routes,  the  timeliness  of  its  deliveries,  its 
ability  to  handle  the  variety  of  products  involved,  and  the  manner  in  which  it 
degrades  under  unfavorable  environmental  pressures.  This  functional  approach 
cannot  be  undertaken  by  isolating  the  communications  problem  from  the  environ¬ 
mental  and  problem-solving  tasks  of  the  organization.  For  this  reason,  the  research 
required  will  necessarily  be  more  integrative  and  problem-oriented.  Previous  stud¬ 
ies  using  such  an  integrative  approach  indicate  the  promise  of  this  research  strat¬ 
egy  (Chapanis,  1975;  Hiltz,  1975;  Hiltz,  Johnson,  and  Agle,  1978). 

In  command  and  control  centers  performing  tactical  decisionmaking,  sensor 
and  intelligence  reports  arrive  at  variable  rates  and  in  different  formats,  and  they 
must  be  integrated  into  a  coherent  interpretation.  Several  people  cooperate  to 
integrate  and  process  data  reports,  and  they  have  extremely  poor  ways  to  convey, 
fuse,  or  contrast  their  personal  world  views.  The  Plexiglas  status  board  and  the 
manual  plotting  boards  of  much  of  today’s  Navy  are  inadequate  technologies  for 
supporting  timely  fusion  of  sensor  data,  and  they  raise  several  crucial  communica¬ 
tions  questions:  How  can  two  or  more  people  contribute  to  a  common  representa¬ 
tion  that  supports  both  individual  and  integrative  perspectives?  How  can  hypothe¬ 
ses  about  events  or  objects  autonomously  revise  themselves  as  time  progresses? 
How  can  tasks  be  divided  among  people  to  promote  early  attention  to  the  most 
crucial  and  perishable  data? 

Experiments  that  vary  communication  structures  for  such  tasks  would  neces¬ 
sarily  also  vary  task  structure  and  decision  strategies  (see  Sections  5.6.1  and  5.6.2). 
Hierarchies  may  prove  effective  for  these  tasks,  but  some  previous  research  sug- 
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gests  that  flat  organizations  with  direct  communications  among  adjacent  spatial 
sectors  are  preferable  (Hayes-Roth  and  Wesson,  1978).  However,  studies  need  to 
investigate  tasks  with  realistic  data  volumes.  Stressing  a  team  with  a  heavy  work¬ 
load  will  expose  the  interactions  among  volume,  uncertainty,  perishability,  ease  of 
communication,  and  organization.  Analyses  of  communication  failures  should  as¬ 
sist  in  early  theoretical  development  of  communications  models  for  such  tasks. 
Experiments  that  vary  organizational  structure  by  partitioning  it  to  support  spa¬ 
tially  or  temporally  defined  subtask  groups  should  help  elucidate  the  relationship 
between  communication  and  sharing  of  related  data.  Finally,  related  research 
could  profitably  include  studies  of  special  interfaces  for  manipulating  geographic 
data  interactively  and  cooperatively  among  two  or  more  co-workers  (see  Section 
5.8.3). 

This  research  requires  a  laboratory  simulation  that  allows  several  situation 
assessors  to  receive  and  process  simulated  sensor  reports  and  to  manipulate  them 
geographically,  interactively,  and  cooperatively.  Experts  in  military  situation  as¬ 
sessment,  graphics,  teleconferencing,  interactive  maps,  cooperative  problem-solv¬ 
ing,  communication,  and  experimental  methodology  should  collaborate  in  this  re¬ 
search. 

In  the  future,  decisionmaking  teams  will  be  crucially  dependent  on  the  ability 
to  rapidly  integrate  combat  data  from  numerous  input  sources.  Current  training 
on  these  tasks  is  limited  by  the  lack  of  a  theory  of  communications  that  might 
expedite  and  facilitate  data  aggregation.  Communication  in  this  environment  en¬ 
compasses  what  needs  to  be  conveyed,  to  whom,  by  what  time,  and  in  what  format. 
Improved  understanding  of  these  issues  will  have  dramatic  effects  on  systems 
acquisitions,  training,  and,  most  important,  force  effectiveness. 


5.7.  IMPROVEMENT  OF  TRAINING  FOR  TEAMS  AND  TEAM 
MEMBERS 

This  report  has  developed  a  view  of  decisionmaking  teams  as  groups  of  cogni¬ 
tive  specialists  whose  efficient,  coordinated  performance  is  enhanced  by  extensive 
practice  and  interfered  with  by  personnel  turnover.  This  view  suggests  several 
opportunities  for  training  to  enhance  both  the  cognitive  and  cooperative  aspects  of 
performance.  These  promising  training  approaches  are  described  below. 

5.7.1.  "Intelligent”  Computer-Assisted  Instruction  for  Team 
Training18 

In  the  classroom,  in  training  exercises,  and  in  on-the-job  training,  an  instructor 
has  several  important  roles.  These  include  monitoring  the  growth  of  each  trainee’s 
knowledge  and  skills,  providing  feedback  on  performance,  and  giving  advice  on 
how  to  best  use  the  available  resources.  Instructor  skill  varies  widely  and  instruc¬ 
tion  is  considered  by  many  to  be  more  an  art  than  a  science.  Over  the  past  decade, 
research  has  investigated  complex  learning  and  instruction  using  adaptive  comput¬ 
er-based  instructional  systems.  This  work  has  made  progress  in  identifying  and 
formalizing  the  components  of  the  instructor’s  role  as  tutor  and  coach.  The  result- 
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in g  instructional  systems  have  been  called  "intelligent”  CAI  (ICAI),  both  to  indicate 
their  connection  to  other  artificial  intelligence  research  (see  Clancey  and  Bennett, 
1979,  for  the  AI  perspective)  and  to  differentiate  them  from  "frame-based”  CAI,  an 
approach  that  utilizes  prestored,  static  data  structures  for  all  instructional  materi¬ 
als  and  decisions. 

Research  involving  ICAI  systems  serves  three  important  functions:  (1)  it  per¬ 
mits  exploration  of  performance  models,  learning  models,  and  instructional  strate¬ 
gies  and  methodologies  in  general;  (2)  it  supports  development  of  operational  ICAI 
systems  that  could  standardize  tutoring  and  coaching  skills;  and  (3)  it  offers  the 
promise  of  constant  availability  of  instruction  for  trainees  despite  the  incipient 
shortage  of  human  instructors  in  the  Navy.  This  shortage  is  particularly  acute  in 
the  context  of  team  training,  since  it  would  be  desirable  to  have  one  instructor  for 
each  trainee. 

Previous  ICAI  research,  supported  primarily  by  ONR,  has  focused  on  individ¬ 
ual  instruction  on  facts  (Collins,  1975)  and  problem-solving  procedures  (Brown  and 
Burton,  1975;  Carr,  1977;  Bates,  1978;  Burton  and  Brown,  1979;  Clancey,  1979). 
These  studies  have  led  to  the  understanding  of  how  to  represent  explicitly  (1)  the 
structure  of  domain  (task)  knowledge  and  skills  for  instructional  purposes  and  (2) 
general  tutoring  and  coaching  strategies.  Although  ICAI  systems  have  demon¬ 
strated  the  feasibility  of  the  instructional  principles  and  of  system  design,  evalu¬ 
ation  of  their  effectiveness  for  training  has  been  limited.  Exploratory  development 
by  the  Navy  of  prototype  ICAI  training  systems  has  begun  at  the  Naval  Personnel 
Research  and  Development  Center  (San  Diego)  and  is  being  contemplated  by  the 
Naval  Training  Equipment  Center  (NTEC)  in  Orlando.  However,  this  technology 
has  not  yet  been  applied  to  team  training. 

New  research  should  explore  the  utility  of  ICAI  for  training  cognitive  skills  in 
team  decisionmaking.  The  focus  should  be  on  identifying  and  representing  (1)  the 
knowledge  each  team  member  acquires  about  the  task  and  about  the  competence 
and  interaction  styles  of  other  team  members  (see  Sections  5.7.2  and  5.7.3),  (2)  the 
processes  that  use  this  knowledge  to  produce  individual  performance,  and  (3) 
instructional  strategies  for  the  development  of  those  performance  skills.  The  sys¬ 
tem  architecture  should  embody  a  simulation  of  a  team  exercise  in  which  the 
performance  of  one  or  more  team  members  is  dynamically  simulated  in  interaction 
with  a  trainee.  Instructional  strategies  to  be  explored  would  include  simulation 
manipulation  (e.g.,  systematic  alteration  of  simulated  team  members’  behaviors) 
and  scheduling  of  tutor  intervention  within  the  simulation  exercises.  The  research 
should  begin  with  the  analysis  and  empirical  study  of  team  training  followed  by 
incremental  implementation  of  the  ICAI  system  based  on  the  results  of  that  study. 
The  theoretical  ideas  and  ICAI  system  capabilities  would  be  refined  through  cycles 
of  prototype  development  and  experimental  testing. 

This  approach  follows  previous  examples  of  ICAI  research  directly,  although 
none  of  the  prior  work  has  examined  team  performance  and  training.  Team  mem¬ 
ber  simulation  is  currently  being  pursued  at  NTEC,  where  pilot  behavior  is  simulat¬ 
ed  in  several  air  control  training  tasks.  However,  that  research  does  not  address 
questions  of  how  to  manipulate  simulated  performance  for  instructional  purposes. 

The  proposed  research  is  relatively  expensive  in  terms  of  time,  equipment,  and 
manpower.  The  first  exploratory  development  of  an  ICAI  system  for  individual  skill 
training  has  just  begun,  after  a  decade  of  research  on  ICAI.  The  development  effort 


45 


will  probably  require  five  years  to  produce  a  prototype  suitable  for  field  evaluation, 
and  investigations  of  ICAI  for  teams  will  probably  require  five  years  to  provide  a 
basis  for  exploratory  prototype  development.  The  project  will  require  systematic 
observation  of  operational  team  training  techniques  and  training  experiments  in¬ 
volving  the  experimental  ICAI  system.  Requisite  computing  capabilities  include  a 
high-level  artificial  intelligence  programming  language  (e.g.,  LISP,  Smalltalk),  a 
dedicated  time-sharing  research  machine  (e.g.,  DEC20  or  VAX/UNIX),  and/or  a 
network  of  powerful,  single-user  systems  (e.g.,  PERQs,  Altos).  The  research  staff 
should  include  cognitive  psychologists,  artificial  intelligence  researchers,  and  edu¬ 
cational  psychologists  or  training  specialists. 

The  long-term  payoff  of  the  research  would  better  training  for  team  members 
with  a  reduction  in  demands  on  human  instructors.  The  proposed  research  is, 
however,  relatively  risky,  given  its  cost.  While  the  theoretical  and  empirical  studies 
will  expand  our  understanding  of  team  processes,  the  implementation  of  the  experi¬ 
mental  ICAI  system  to  test  and  refine  that  understanding  will  be  an  ambitious 
undertaking. 

5.7.2.  Training  Mental  Models19 

Section  5.4.2  discusses  the  importance  of  the  "mental  models”  individuals  bring 
to  bear  on  a  problem  and  the  potential  interactions  among  different  mental  models 
held  by  team  members.  To  control  such  interactions,  it  may  be  desirable  to  train 
prospective  team  members  to  use  specific  mental  models.  For  example,  consider  the 
case  in  which  a  new  individual  enters  into  a  previously  stable  team.  The  original 
team  members  may  have  evolved  a  characteristic  approach  to  their  task.  If  the  new 
individual’s  approach  or  conception  of  the  task  differs,  this  may  degrade  both  team 
performance  and  social  interactions  among  team  members.  Training  the  new  indi¬ 
vidual  in  the  team’s  approach  might  circumvent  some  of  these  difficulties.  Con¬ 
versely,  for  other  tasks  it  may  be  desirable  for  individuals  to  have  different  mental 
models.  This  could  give  the  team  the  flexibility  of  different  perspectives  in  handling 
novel  problems.  Again,  it  may  be  possible  to  train  individuals  to  use  specific  mental 
models,  thus  facilitating  the  composition  of  effective  teams. 

There  are  several  possible  approaches  to  the  training  of  mental  models.  For 
example,  one  might  formulate  a  model  in  operational  terms  and  communicate  it 
directly  to  students.  The  formalization  might  include  a  statement  of  acknowledged 
goals  and  the  relationships  among  them,  preferred  strategies  and  methods,  pre¬ 
sumed  relationships  among  entities  under  consideration,  and  so  on.  Alternatively, 
one  might  attempt  to  teach  models  by  analogy.  Ideally,  the  analogous  problem 
would  be  one  that  was  quite  familiar  to  the  student  and  which  the  student  would 
reliably  approach  systematically.  Given  appropriate  instructions  and  the  opportu¬ 
nity  to  work  on  both  kinds  of  problems  in  close  succession,  the  student  could  learn 
to  transfer  his  or  her  approach  to  the  familiar  problem  to  the  target  problem. 

Prior  research  has  successfully  applied  both  approaches  to  the  training  of 
simple  problem-solving  and  learning  strategies  (Hayes-Roth,  1980;  Reed,  Ernst,  and 
Banerji,  1974;  Thorndyke  and  Stasz,  1980).  These  and  other  studies  (e.g.,  Amarel, 
1968;  Bruner,  Goodnow,  and  Austin,  1956;  Greeno  and  Simon,  1974;  Newell  and 
Simon,  1972;  Simon,  1976)  suggest  that  human  problem-solving  is  inherently  flexi- 
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ble  and  amenable  to  modification.  However,  comprehensive  mental  models  appro¬ 
priate  for  complex  decisionmaking  control  tasks  would  be  considerably  more  com¬ 
plex  than  the  strategies  previously  studied. 

An  initial  investigation  of  these  problems  should  address  the  following  research 
questions: 

1.  What  functional  units  adequately  operationalize  people’s  mental  models? 

2.  How  can  we  communicate  these  functional  units? 

3.  What  familiar  problem-solving  situations  represent  useful  mental  models? 

4.  How  can  an  individual’s  mental  model  be  diagnosed? 

5.  What  is  the  most  effective  way  to  present  analogical  problem-solving 
experiences  to  trainees? 

6.  Under  what  circumstances  do  these  alternative  methods  succeed? 

7.  What  other  training  methods  are  feasible? 

8.  What  problems  arise  as  the  model  to  be  trained  becomes  more  complex? 

Formalizing  and  communicating  such  complex  models  will  pose  formidable 
challenges  for  cognitive  and  instructional  psychologists.  Therefore,  research  ex¬ 
ploring  the  training  of  mental  models  must  be  considered  a  high-risk  effort.  On  the 
other  hand,  the  potential  payoff  is  also  high  and  extends  beyond  the  immediate 
consideration  of  team  performance.  The  basic  research  questions  bear  directly  on 
many  other  important  areas  including  computer  simulation  of  real  systems,  com¬ 
munication  of  complex  information  structures  to  humans,  forecasting,  planning, 
and  decisionmaking. 

5.7.3.  Training  Empathetic  tyodeis20 

As  team  members  develop  mental  models  of  the  team  task,  they  also  learn 
about  the  roles  others  play  in  the  team  and  about  those  other  individuals  as  people. 
A  promising  line  of  research  complementing  work  on  mental  models  would  formal¬ 
ize  these  social  observations  for  training  purposes  and  would  examine  their  utility 
in  improving  performance,  reducing  the  negative  effects  of  turnover,  and  increas¬ 
ing  motivation  and  morale  of  team  members. 

Research  in  social  psychology  has  indicated  that  knowledge  of  the  roles  of  other 
team  members  can  both  increase  understanding  of  the  problems  seen  from  others’ 
perspectives  and  improve  overall  team  performance.  New  research  is  needed  on 
the  utility  of  learning  the  roles  of  others  in  the  team  training  situation.  Specifically, 
empathetic  models  might  be  incorporated  into  the  training  environment  after  indi¬ 
viduals  have  learned  their  own  particular  job  skills.  Team  members  might  practice 
their  skills  in  teams  with  other  members  (real  or  simulated)  who  are  performing 
suboptimally.  Such  manipulations  will  require  an  understanding  of  potential  prob¬ 
lem  situations  and  errors  occurring  on  particular  tasks.  This  training  should  teach 
team  members  to  adapt  their  own  performance  based  on  variability  in  their  co¬ 
horts’  performance. 

Such  research  may  lead  to  a  more  general  exploration  of  teamwork  skills.  Team 
members  may  learn  not  only  what  functions  others  perform  but  also  idiosyncratic 
personal  knowledge  about  others.  Can  one  learn,  through  this  idiosyncratic  knowl¬ 
edge,  general  knowledge  applicable  to  adapting  to  others?  One  research  approach 
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would  be  to  develop  flexibility  exercises  in  which  individuals  are  presented  with 
hypothetical  problems  in  team  situations  involving  the  need  to  adapt  to  the  behav¬ 
ior  of  some  other  team  member  (see  Section  5.7.4).  The  kinds  of  information  that 
can  be  translated  into  general  adaptation  skills  can  be  assessed  by  measuring 
performance  on  new  hypothetical  problems  of  a  different  type. 

It  may  also  be  possible  to  develop  empathetic  exercises  to  deal  with  particular 
chronic  team  problems.  Role-playing  techniques  and  even  paper-and-pencil  empa¬ 
thy-inducing  techniques  increase  an  individual’s  ability  to  take  the  role  of  another 
and  understand  that  person’s  situation.  A  target  problem  that  might  be  addressed 
by  such  a  methodology  is  the  variation  in  performance  demands  of  different  team 
members.  If  unstressed  team  members  were  to  take  the  role  of  particularly  over¬ 
loaded  cohorts,  some  of  the  problems  facing  other  members  might  become  more 
salient  and  better  understood  by  all.  This,  in  turn,  might  improve  the  interpersonal 
climate  of  the  teams. 

Experimental  research  on  empathetic  models  could  be  readily  incorporated 
into  the  Research  Center  laboratory.  The  identification,  development,  and  experi¬ 
mental  evaluation  of  training  problems  and  models  would  require  the  collaboration 
of  cognitive  and  social  psychologists.  If  such  models  and  training  methods  were 
automated,  additional  research  would  require  expertise  in  artificial  intelligence 
and  computer  science. 

The  ability  to  train  empathetic  models  will  serve  several  functions.  First,  such 
training  might  improve  team  performance  by  broadening  team  members’  under¬ 
standing  of  the  overall  task.  Second,  it  might  help  compensate  for  disruptive  effects 
of  turnover  by  facilitating  the  incorporation  of  new  team  members  (see  Section 
5.6.2).  And  finally,  it  might  improve  morale  by  reducing  friction  and  promoting 
esprit  de  corps. 

5.7.4.  Training  Flexibility  in  SOP-Based  Performance21 

Typically,  team  members  perform  according  to  well-learned  SOPs,  which  en¬ 
able  individuals  and  teams  to  respond  to  situations  rapidly  and  consistently.  Thus, 
the  Navy  has  sought  to  institute  SOPs  for  both  individuals  and  teams  whenever 
sufficiently  detailed  task  analyses  are  available. 

A  large  proportion  of  training  time  is  spent  drilling  SOPs  in  standard  exercises. 
Yet  in  many  situations,  such  rote  performance  may  be  inappropriate  or  even 
disastrous.  For  example,  when  systems  malfunction,  team  members  must  quickly 
decide  what  to  do  and  must  coordinate  their  unrehearsed  actions.  As  another 
illustration,  wartime  incapacity  of  team  members  (or  simply  their  impromptu  re¬ 
placement)  demands  flexibility  as  the  team  must  cope  with  changing  personnel 
resources.  Finally,  enemy  tactics  may  differ  from  those  that  training  can  reason¬ 
ably  anticipate. 

Prior  research  on  the  development  of  automaticity  in  skilled  performance  may 
provide  insights  into  the  question  of  how  to  enable  flexible  responses  despite  the 
existence  of  highly  overlearned  procedures.  Most  of  this  research,  however,  has 
considered  rote  laboratory  learning  tasks  and  motor-skill  learning,  not  complex 
cognitive  procedures.  Research  has  indicated  that  the  inappropriate  application  of 
SOPs  is  responsible  for  many  aircraft  emergencies  and  accidents,  incidents.  Initial 
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research  on  how  to  train  aircrews  to  maintain  flexibility  has  already  provided  some 
insights  of  general  value  {Lucaccini,  1978). 

New  research  should  develop  flexible  training  procedures  for  decisionmaking 
teams  and  should  test  their  effectiveness  in  standard,  unusual,  and  totally  unre¬ 
hearsed  situations.  Several  questions  about  the  implications  of  training  SOPs  need 
to  be  investigated: 

1.  An  SOP  (or  set  of  SOPs)  may  be  valid  for  only  a  subset  of  the  situations 
that  require  some  response.  If  the  exceptional  circumstances  are  rare, 
they  may  not  be  recognized  until  the  SOP  is  invoked,  leading  to  undesir¬ 
able  consequences.  What  training  could  retain  the  benefits  of  SOPs,  yet 
enable  the  recognition  and  flexible  response  to  situations  for  which  avail¬ 
able  SOPs  are  inappropriate?  Do  prior  experiences  with  unusual  situa¬ 
tions  and  strong  cognitive  skills  assist  in  this  training? 

2.  Teams  may  evolve  and  institute  unique  SOPs  that  reflect  their  specific 
environment  and  individual  skills.  What  are  the  implications  of  institu¬ 
tionally  imposed  SOPs  for  the  development  of  these  evolved  SOPs?  What 
are  the  implications  of  evolved  SOPs  for  integrating  new  team  members 
trained  in  institutional  SOPs? 

3.  In  decisionmaking  teams,  the  number  and  specificity  of  SOPs  vary  from 
job  to  job.  How  does  the  more  constrained  behavior  of  members  who 
follow  detailed  SOPs  affect  the  performance  of  members  who  must  re¬ 
spond  flexibly  in  their  roles?  Do  only  high-level  decisionmakers  require 
training  flexibility,  or  do  low-level  operators  require  such  training  as  well? 

New  research  to  investigate  these  questions  would  require  long-term  access  to 
subject  teams,  because  of  the  time  needed  to  train  highly  automatic  execution  of 
SOPs  and  to  enahle  teams  to  evolve  their  own  SOPs.  Either  comparisons  of  training 
techniques  or  intensive,  longitudinal  studies  of  a  few  teams  could  be  undertaken 
to  draw  limited  generalizations  about  the  answers  to  the  research  questions  posed 
above. 


5.7.5.  Dynamic  Gaming  as  a  Training  Aid*1 

Military  gaming  has  a  long  history  as  a  training,  research,  and  planning  tech¬ 
nique  (Young,  1959;  Weiner,  1964;  Hansrath,  1971).  The  simulation  of  combat 
interactions  between  opposing  forces  offers  the  potential  for  exploring  a  wide  vari¬ 
ety  of  team  activities  sifch  as  planning,  allocation,  decisionmaking,  and  responding 
to  a  spectrum  of  adversary  actions.  In  recent  years,  considerable  emphasis  has  been 
placed  on  the  use  of  gaming  as  a  research  method  for  examining  force  structure, 
hardware,  plans,  and  tactical  issues  (see  Brewer  and  Shubik,  1979).  Similarly, 
computer-controlled  dynamic  gaming  is  being  increasingly  used  to  train  military 
personnel  in  hardware-rich  simulators  (e.g.,  the  Naval  Electronic  Warfare  Simula¬ 
tion  (NEWS),  the  Battalion  Analyzer  and  Tactical  Trainer  for  Local  Engagements 
(BATTLE),  and  the  Combined  Arms  Tactical  Training  Simulator  (CATTS)  of  the 
Army). 
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Despite  these  applications,  there  are  several  major  issues  in  the  use  of  dynamic 
gaming  as  a  training  technique  that  have  received  little  attention.  These  include: 

1.  What  do  participants  actually  learn  from  games? 

2.  How  relevant  is  the  acquired  knowledge  to  actual  military  operations? 

3.  Can  dynamic  gaming  provide  training  on  team  activities  that  cannot  be 
carried  out  effectively  by  other  techniques? 

Little  is  known  of  the  utility  of  dynamic  gaming  as  a  team  training  technique. 
Generally,  evaluation  of  the  utility  of  games  as  training  devices  rests  on  subjective 
estimates  by  the  participants  rather  than  on  measures  of  knowledge  transfer  to 
operation  settings.  Furthermore,  such  training  has  generally  emphasized  individ¬ 
ual  learning  rather  than  team  learning. 

Research  should  investigate  the  operational  situation  and  the  development  of 
principles  for  the  effective  use  of  gaming  in  team  training.  New  research  should 
undertake  two  activities: 

1.  An  evaluation  of  current  training  methods  employing  gaming  and  simula¬ 
tion  in  terms  of  such  variables  as  motivation,  knowledge  acquisition,  deci¬ 
sionmaking,  and  realism. 

2.  The  development  of  a  series  of  experimental  games  that  highlight  specific 
training  objectives  such  as  assimilating  complex  information  structures, 
handling  uncertainty  in  decisionmaking  information,  rapid  response  with 
incomplete  knowledge,  and  adaptation  to  new  allocation  options.  These 
experimental  games  would  initially  represent  simple  dynamic  situations 
involving  team  participation.  They  could  be  used  to  test  hypotheses  about 
goal  perception,  task  models,  team  structure,  option  generation,  and  flexi¬ 
bility  of  responses.  Subsequent  efforts  could  consider  more  complex  situa¬ 
tions  with  more  typical  operational  characteristics. 

Research  efforts  could  combine  evaluations  of  current  gaming  activities  (focUh- 
ing  on  their  specific  training  objectives  and  methodology)  and  development  of  new 
experimental  training  games.  A  one-  to  two-year  effort  should  be  considered  for 
these  efforts.  Further  effort  should  be  contingent  on  a  review  of  the  utility  of 
gaming  techniques  for  team  training  research. 

The  value  of  this  research  would  lie  primarily  in  the  development  of  effective 
gaming  techniques  for  training  real-time  decisionmaking.  In  particular,  it  would 
lead  to  the  development  of  planning,  allocation,  and  decision  aids  for  decisionmak¬ 
ing  teams.  The  main  thrust  of  the  research  effort  should  be  to  provide  a  sound 
conceptual  basis  for  the  prescriptive  use  of  gaming  for  team  training,  as  opposed 
to  the  use  of  "face  validity”  as  a  rationale  for  dynamic  gaming  techniques. 

5.7.6.  Enriching  Feedback  in  Training  and  Operational 
Environments*3 

Feedback  is  a  critical  component  of  training.  Without  it,  individuals  cannot 
learn  their  responsibilities  as  individual  performers  and  as  team  members.  Periodic 
evaluation  and  feedback  in  the  operating  environment  is  also  required  for  mainte- 
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nance  of  skilled  performance  and  high  motivation.  At  present,  feedback  in  team 
training  and  in  the  operating  environment  tends  to  be  infrequent,  irregular,  and 
often  subjective.  Training  practices  should  include  rapid,  useful  feedback  as  an 
integral  part  of  each  practice  session.  Hence,  research  aimed  at  enriching  feedback 
has  the  potential  for  improving  team  performance  and  motivation. 

Previous  experimental  research  has  established  several  principles  governing 
the  relationship  between  feedback  and  performance  (see  reviews  by  Klaus  and 
Glaser,  1970;  Nadler,  1979).  These  principles  include  the  following: 

1.  Feedback  should  be  as  temporally  proximal  to  performance  as  possible. 

2.  Positive  feedback  benefits  performance  and  satisfaction,  except  when  indi¬ 
viduals’  errors  are  obscured  by  the  positive  feedback. 

3.  Individual  and  team  feedback  are  both  important,  since  they  serve  differ¬ 
ent  functions.  Positive  group  feedback  enhances  team  spirit  and  group 
morale.  Individual  feedback  enhances  motivation  and  provides  clues  as  to 
how  to  correct  performance. 

4.  Negative  feedback  to  either  the  group  or  the  individual  has  mixed  effects. 
Under  some  circumstances,  motivation  and  performance  decrease  and 
defensiveness  increases. 

5.  "Process-oriented”  feedback  (i.e.,  generalized  feedback  about  the  group 
environment  rather  than  the  group  product)  may  reduce  defensiveness 
but  may  not  improve  performance. 

These  considerations  generate  the  requirement  for  new  research  on  feedback. 
Such  research  includes  field  studies  and  experimental  studies,  as  summarized 
below. 

Field  studies  should  assess  Navy  team  members’  desires  and  requirements  for 
feedback.  Where  do  they  feel  they  need  feedback?  At  what  points  during  perfor¬ 
mance  do  they  want  feedback?  Do  they  desire  additional  group  feedback  or  individ¬ 
ual  feedback?  How  do  they  feel  it  can  best  be  integrated  into  the  task  environment 
without  obstructing  ongoing  performance?  Research  in  this  area  should  focus  on 
whether  the  perceived  need  for  feedback  stems  from  uncertainty  about  one’s  own 
performance  or  a  need  for  approval  and  support. 

Experimental  work  should  address  the  following  high-priority  issues: 

1.  In  most,  if  not  all,  previous  studies,  an  experimenter  or  another  agent  outside 
the  team  provides  feedback.  However,  several  interesting  questions  arise  in  the 
consideration  of  who  provides  the  feedback.  Should  feedback  be  centralized  or 
decentralized?  If  it  should  be  centralized,  should  it  come  from  outside  the  team  or 
from  the  team  leader?  Can  team  members  reinforce  each  other?  Can  machines 
provide  individual  and  team  feedback?  If  so,  under  what  circumstances  is  such 
feedback  reinforcing? 

2.  In  team  situations,  there  are  multiple  options  for  supplying  feedback  during 
performance.  Research  should  investigate  the  conditions  under  which  individual 
versus  overall  team  feedback  is  most  effective.  When  the  team  is  performing,  it  may 
be  difficult  to  provide  feedback  to  either  individuals  or  teams  without  interrupting 
the  time-course  of  the  scenario.  Research  should  explore  options  for  overcoming 
these  timing  problems.  Such  options  may  include  switching  simulated  team  mem¬ 
bers  into  the  system  while  feedback  is  given,  multiplexing  feedback  with  continuing 
performance,  or  videotaping  performance  for  post-training  review  and  evaluation. 
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Finally,  research  should  investigate  whether  feedback  should  optimally  be  pro¬ 
vided  in  public  or  privately. 

3.  Negative  feedback  is  problematic  but  is  essential  to  improving  performance. 
What  is  the  best  way  to  provide  negative  feedback?  Should  negative  feedback  be 
provided  only  on  a  private  basis  and  only  to  individuals?  Can  it  be  provided  by  a 
machine?  Who  should  provide  negative  feedback — the  leaders  or  other  group  mem¬ 
bers?  Is  negative  feedback  to  the  overall  team  desirable  if  performance  deficits 
stem  from  poor  coordination? 

4.  To  be  effective,  feedback  must  include  a  usage  component.  That  is,  the 
recipient  must  be  able  to  convert  the  feedback  into  the  appropriate  task  behaviors. 
One  approach  would  be  to  study  how  feedback  is  used  when  it  is  provided  in  the 
naturally  occurring  environment,  and  how  its  use  can  be  increased.  Experimental 
studies  that  systematically  manipulate  message  characteristics  of  the  feedback 
would  also  seem  well  suited  to  addressing  this  problem. 

5.  Experimental  studies  should  also  focus  on  the  role  of  feedback  over  time  to 
assess  the  impact  of  feedback  both  on  teams  that  are  performing  very  well  and  on 
those  that  are  functioning  poorly.  The  ability  to  alter  a  teams’  performance  and 
esprit  de  corps  over  time  may  depend  on  the  complex  interaction  between  baseline 
performance  and  the  type  of  feedback  it  receives. 

This  research  would  require  the  collaboration  of  social  psychologists,  experi¬ 
mental  psychologists,  instructional  or  educational  psychologists,  and  perhaps  com¬ 
puter  scientists.  Studies  could  be  conducted  both  in  the  research  laboratory  and  at 
Navy  training  sites.  Because  of  the  importance  of  feedback  in  training  and  the 
current  deficiencies  in  the  operational  administration  of  feedback,  this  research 
could  have  tremendous  benefits  for  team  performance. 


5.8.  MAN-MACHINE  SYSTEMS  FOB  TASK  PERFORMANCE 

Traditionally,  systems  involved  in  team  performance  have  been  supported  by 
two  kinds  of  interaction  between  humans  and  machines:  In  the  first,  humans  use 
of  the  computer  as  a  data-processing  medium  (Hiltz  and  Turoff,  1978).  In  the 
second,  they  monitor  and  control  a  complex,  usually  monolithic  computer  system. 
However,  team  decisionmaking  requires  a  more  integrative  approach  to  man- 
machine  cooperation.  Each  member  of  the  interaction  brings  unique  capabilities  to 
the  task,  but  little  is  known,  theoretically  or  practically,  about  how  to  decompose 
team  tasks  or  how  to  allocate  subtasks  to  humans  and  machines.  Furthermore,  we 
know  very  little  about  conveying  what  a  human  knows  or  believes  to  a  machine 
or,  conversely,  transforming  a  machine  representation  of  a  complex  situation  into 
a  form  readily  accessible  to  a  human.  Several  approaches  to  the  design  of  man- 
machine  systems  could  improve  both  the  efficiency  and  quality  of  team  perfor¬ 
mance. 

5.8.1.  Use  of  Automated  Specialists  to  Supplement  or  Replace 
Team  Members24 

Team  performance  is  currently  limited  by  the  difficulty  of  arranging  training 
exercises.  The  magnitude  of  the  operation  required  for  training  exercises  could  be 
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reduced  by  creating  environments  in  which  a  single  team  member  could  interact 
with  intelligent  simulations  of  other  team  members.  Furthermore,  requirements 
for  human  participation  in  actual  operations  might  be  substantially  altered  by 
increasing  reliance  on  such  automated  simulations  of  team  members.  Both  of  these 
goals  require  research  to  transfer  human  capabilities  to  machine-based  specialists. 

Prior  research  has  succeeded  in  automating  human  expertise  in  limited  appli¬ 
cations.  This  work  has  focused  chiefly  on  electric  circuit  diagnosis  (Brown  and 
Burton,  1975),  medical  diagnosis  (ShortlifFe,  1976)  air  traffic  control  (Klass,  1979), 
and  speech  and  image  processing  (Mostow  and  Hayes-Roth,  1978).  Such  research 
typically  begins  with  a  cognitive  analysis  of  task  performance  (see  Section  5.4.1). 
This  analysis  ordinarily  requires  thinking-aloud  protocols  produced  during  actual 
task  performance,  the  construction  of  a  model  that  accounts  for  the  protocols,  the 
implementation  of  a  computer  simulation  of  the  model,  and  finally,  the  refinement 
of  the  knowledge  and  strategies  in  the  simulation. 

Problem-solving  in  decisionmaking  teams  seems  ideally  suited  to  this  research 
approach.  The  performance  of  team  members  should  be  observed,  modeled,  simu¬ 
lated,  and  refined  in  a  laboratory  or  training  setting.  Then,  the  derived  machine- 
based  team  members  may  be  used  in  a  variety  of  experimental  ways  to  determine 
their  benefits  and  costs.  First,  the  performance  of  the  models  can  be  compared  to 
that  of  their  human  counterparts.  Second,  the  models  can  provide  surrogates  for 
missing  human  members  in  training  contexts,  as  suggested  in  Section  5.3.3.  Simu¬ 
lated  team  members  might  include  either  individuals  on  the  team  or  individuals 
outside  the  team  who  interact  with  team  members.  These  simulations,  if  successful, 
may  also  provide  standalone  alternatives  to  humans  for  certain  tasks.  Finally,  the 
models  may  be  used  by  human  team  members  as  computational  aids  for  task 
performance  (see  Section  5.8.2). 

Beyond  the  normal  difficulties  associated  with  building  heuristic  models,  this 
area  of  application  \yill  require  careful  attention  to  the  cooperative  and  adaptive 
aspects  of  team  behavior.  The  computer  models  will  need  to  be  able  to  accept 
questions  and  tasks  from  others,  to  convey  tasks  and  information  to  others,  and  to 
explain  and  rationalize  their  behavior.  Further,  these  specialists  may  need  to  dem¬ 
onstrate  flexibility  in  responses  to  unexpected  situations  (see  Section  5.7.4). 

This  research  will  require  the  combined  efforts  of  psychologists,  who  will  per¬ 
form  task  analysis  and  initial  model  building,  and  computer  scientists,  who  will 
build  and  refine  the  machine  implementations,  initial  performance  should  be  moni¬ 
tored  in  team  problem-solving  experiments  in  the  research  laboratory. 

This  research  offers  significant  promise  for  improving  team  performance  by 
creating  new  training  opportunities.  Currently,  team  training  requires  simultane¬ 
ous  participation  of  numerous  human  team  members.  By  creating  machine-based 
members,  training  will  be  more  accessible,  less  expensive,  and  more  controllable. 
This  in  turn  will  increase  the  quality  and  quantity  of  team  training  each  member 
receives.  It  may,  in  addition,  obviate  the  need  for  special  training  facilities,  since 
the  space  requirements  for  training  can  be  reduced  to  that  required  to  support  the 
number  of  humans  who  will  be  trained  simultaneously.  Furthermore,  the  research 
may  lead  to  the  improvement  of  team  skills  through  the  automation  of  both  mun¬ 
dane  and  highly  complex  team  activities. 
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5.8.2.  Automated  Planning  Aids  for  Decisionmaking2* 

Section  5.8.1  discusses  the  development  of  automated  specialists  to  supplement 
team  members.  One  such  aid  deserving  special  attention  would  assist  individuals 
in  the  difficult  and  imprecise  task  of  tactical  planning.  Automated  planning  aids 
could  serve  a  number  of  useful  purposes,  including  the  following: 

1.  Performance  of  Standard  Computational  Functions  (such  as  table  lookup, 
mathematical  calculations,  and  information  retrieval).  This  kind  of  assistance 
would  presumably  improve  both  the  accuracy  and  the  speed  of  computations  under¬ 
lying  plan  development. 

2.  Bookkeeping  Assistance.  When  several  people  cooperate  on  a  single  prob¬ 
lem,  managing  their  interactions  can  become  a  problem  in  itself.  A  computer  aid 
could  greatly  facilitate  these  interactions  by  (a)  providing  an  efficient  communica¬ 
tion  medium  for  team  members,  regardless  of  their  respective  locations;  (b)  main¬ 
taining  a  well-structured,  perspicuous  record  of  the  current  solution,  incorporating 
the  contributions  of  all  team  members;  and  (c)  maintaining  an  historical  record  of 
the  development  and  rationale  behind  solution  elements  and  their  alternatives. 

3.  Evaluation  of  Tentative  Plans.  There  are  two  general  methods  for  evaluat¬ 
ing  tentative  plans.  The  first  is  simply  to  calculate  a  plan’s  "score”  against  each  of 
several  established  criteria  and  aggregate  component  scores.  While  people  can 
perform  this  kind  of  evaluation,  a  computer  aid  could  provide  greater  speed  and 
reliability,  thereby  permitting  comparative  evaluation  of  many  more  alternative 
plans.  The  second  method  is  to  simulate  the  execution  of  a  plan  and  assess  its 
effectiveness,  shortcomings,  side  effects,  etc.  This  method  has  much  greater  poten¬ 
tial  utility  than  the  first  because  (a)  most  planning  problems  lack  a  satisfactory  set 
of  well-defined,  mutually  consistent  evaluation  criteria,  and  (b)  scoring/aggrega¬ 
tion  procedures  may  not  reliably  discriminate  good  plans  from  poor  plans.  The 
second  method  is  also  much  more  difficult  for  people  to  apply  for  themselves. 
Therefore,  computer  aids  can  make  enormous  potential  contributions  in  this  area. 

4.  Participation  in  the  Planning  Process.  An  intelligent  computer  aid  migty 
participate  in  plan  development  in  two  ways.  It  could  oversee  individual  contribu¬ 
tions,  coordinating  them  into  a  comprehensive  plan,  detecting  and  resolving  con¬ 
flicts,  etc.  This  is  an  important  function  because  independently  generated  plan 
components  frequently  produce  interactions  that  are  both  unforeseen  and  difficult 
to  detect.  As  discussed  in  the  previous  section,  the  aid  could  also  be  endowed  with 
planning  skills  and  knowledge  about  the  problem  domain  and  could  effectively  act 
as  another  team  member.  These  skills  and  knowledge  might  well  simulate  those  of 
a  human  being. 

Items  1-4  show  a  progression  toward  increasingly  sophisticated  aids  playing 
increasingly  central  roles  in  the  planning  process.  Item  1  is  quite  feasible,  given 
existing  technology.  While  items  2-4  represent  state-of-the-art  computer  science 
research,  preliminary  research  in  all  three  areas  is  very  promising  (Goldstein,  1980; 
Hayes-Roth,  Hayes-Roth,  Rosenschein,  and  Cammarata,  1979;  Hayes-Roth  and 
Wesson,  1978;  Hiltz  and  Turoff,  1978;  Klahr,  Faught,  and  Martins,  1980;  Wesson, 
1977).  Therefore,  we  view  this  research  as  low-risk,  with  a  high  potential  payoff. 

The  success  of  such  planning  aids  depends  not  only  upon  their  efficacy  in 
performing  the  appropriate  functions,  but  also  upon  several  other  factors  govem- 
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ing  the  readiness  of  humans  to  rely  on  them.  First,  aids  must  be  easy  for  people 
to  learn  and  to  use.  Second,  they  must  be  reliable  and  accurate.  More  importantly, 
the  users  must  have  confidence  in  their  reliability  and  accuracy.  Third,  they  must 
not  threaten  people's  sense  of  autonomy  or  the  structure  of  authority.  Accordingly, 
research  on  the  development  of  planning  aids  should  be  accompanied  by  related 
research  on  human  factors  and  the  cognitive  and  social  factors  influencing  people’s 
use  of  them. 

5.8.3.  Machine  Aids  for  Cooperative  Problem-Solving26 

Humans  and  machines  are  best  suited  to  different  types  of  tasks.  Information¬ 
processing  decisionmaking  tasks  requiring  both  situation  assessment  and  planning 
can  benefit  from  the  relative  advantages  of  both.  Humans,  for  example,  are  superb 
visual  pattern  perceivers;  machines  are  notoriously  weak  in  this  area.  Humans  are 
the  best  sources  of  heuristics  about  how  to  treat  uncertain  information  and  how  to 
draw  inferences.  However,  machines,  once  instructed,  can  perform  such  inferences 
faster  and  more  systematically  than  humans.  How  then  can  these  two  kinds  of 
processors  cooperate  when  several  people  and  several  machines  must  work  to¬ 
gether?  The  problem  faced  in  man-machine  teams  is  to  identify  the  crucial  kinds 
of  information-processing  responsibilities,  to  allocate  responsibilities  to  the  most 
appropriate  resources,  and  to  design  communications  media  that  facilitate  their 
interactions. 

In  the  situation-assessment  portion  of  a  command  and  control  task,  sensor  and 
intelligence  reports  about  past,  current,  and  potential  future  events  arrive  at  varia¬ 
ble  rates.  Maintaining  a  coherent  and  current  picture  of  the  world  requires  time¬ 
indexing  and  the  fusion  of  these  time-synchronized  messages  into  a  comprehensible 
interpretation.  Where  and  how  s&nsor  data  should  be  aggregated  and  processed  is 
a  central  problem.  Because  timeliness  of  processing  is  essential,  communication 
delays  are  serious  (see  Sections  5.6.1  and  5.6.3).  On  the  other  hand,  passing  raw 
sensor  data  directly  to  central  facilities  requires  high  bandwidths  and  can  easily 
lead  to  a  glut  that  may  delay  the  processing  of  perishable  information.  Moreover, 
several  people  must  often  cooperate  to  integrate  and  process  data  reports,  and  they 
may  have  extremely  poor  methods  for  conveying,  fusing,  or  contrasting  their  per¬ 
sonal  world  views.  Messages  among  cooperating  decisionmakers  vary  in  their  time 
criticality  and  in  the  responses  they  demand.  New  research  should  therefore  in¬ 
vestigate  the  utility  of  alternative  modes  of  communication  among  humans  and 
between  humans  and  machines.  Furthermore,  the  distribution  of  tasks — including 
signal  processing,  signature  analysis,  hypothesis  formation,  and  team  scheduling 
responsibilities — among  people  and  machines  is  a  wide-open  issue  of  considerable 
long-term  import. 

Several  additional  problems  arise  in  the  planning  component  of  the  task.  Fore¬ 
casting  probable  outcomes  of  current  situations  requires  a  combination  of  causal 
reasoning,  simulation,  and  non-deterministic  reasoning.  People  are  essential  in  the 
assessment  of  dangers  and  opportunities  in  forecasted  situations,  yet  they  seem  to 
perform  poorly  both  at  look-ahead  calculations  and  at  maintaining  numerous  alter¬ 
natives.  Machines  excel  at  both  types  of  calculations,  but  the  products  of  their 
calculations  comprise  many  alternative  outcomes,  each  with  numerous  assump- 
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tions  and  details  (see  Section  5.8.2).  Experimental  research  should  explore  methods 
for  translating  this  volume  of  data  to  a  form  intended  for  human  planners.  Further¬ 
more,  new  efforts  are  warranted  on  the  related  problem  of  incremental  replanning. 
How  can  complex  plans  be  changed  marginally  to  respond  to  specific  problems 
detected  during  plan  execution?  Can  a  human  suggest  incremental  fixes?  Can  the 
shared  representation  of  a  plan  in  progress  be  graphical?  And  can  a  computer 
compute  possible  outcomes  rapidly  and  display  them  visually? 

The  research  laboratory  for  these  studies  should  allow  several  situation  asses¬ 
sors  to  receive  and  process  reports  and  to  manipulate  them  geographically,  interac¬ 
tively,  and  cooperatively.  Experts  in  military  situation  assessment  and  planning 
tasks,  graphics,  teleconferencing,  interactive  maps,  cooperative  problem-solving, 
communication,  and  experimental  methodology  should  collaborate  in  this  research. 


6.  PROGRAM  IMPLEMENTATION 


The  research  program  described  in  this  report  emphasizes  a  broad,  integrated, 
and  multivariate  approach  to  the  study  of  information-processing  decisionmaking 
teams.  We  believe  that  this  research  can  best  be  carried  out  in  a  small  number 
of  large-budget,  resource-rich,  Team  Performance  Research  Centers.  Such  cen¬ 
ters  will  be  better  equipped  to  address  the  range  of  important  problems  in  team 
performance  than  would  small,  more  narrowly  focused  contract  efforts.  To  carry 
out  the  range  of  studies  critical  to  the  improvement  of  team  performance,  each 
Research  Center  will  require  an  interdisciplinary  staff  of  researchers  engaged  in 
a  variety  of  related  project  activities.  These  activities  will  include  policy  studies 
focused  on  the  operational  Navy  (as  described  in  Section  5.1),  the  development  of 
laboratory  and  experimentation  capabilities  (as  described  in  Sections  5  2  and  5.3), 
and  scientific  research,  including  theoretical  development  and  empirical  studies,  on 
a  subset  of  the  research  problems  described  in  Sections  5A-5.8. 

Based  on  our  analysis  of  the  requirements  for  research  in  the  team  perfor¬ 
mance  program  and  our  observation  of  facilities  with  similar  goals  and  methods, 
we  have  derived  a  tentative  set  of  specifications  for  these  Research  Centers.  These 
specifications  are  rough  guidelines  for  the  capabilities,  facilities,  and  costs  required 
to  successfully  carry  out  the  research  we  have  recommended. 


6.1.  INSTITUTIONAL  REQUIREMENTS 

A  research  institution  capable  of  undertaking  the  long-term  program  we  have 
described  must  have  the  following  characteristics: 

1.  The  capability  to  assemble  and  manage  a  large  interdisciplinary  team  of 
researchers  with  expertise  in  military  operations  research,  simulation, 
gaming,  cognitive  modeling,  social  psychology,  policy  analysis,  systems 
design,  computer  hardware,  artificial  intelligence,  human  factors,  and  in¬ 
struction. 

2.  Researchers  who  are  available  to  commit  significant  amounts  of  time  to 
the  research. 

3.  The  space,  knowledge,  and  administrative  support  to  create  and  manage 
a  large  computer-based  laboratory  that  has  state-of-the-art  hardware  and 
software,  a  pool  of  subjects  who  are  available  over  a  long  period  of  time, 
and  facilities  for  observing  experiments  within  the  laboratory  (laboratory 
requirements  and  costs  are  discussed  below). 

4.  The  expertise  to  conduct  research  on  combat  operations  and  defense- 
related  command,  control,  and  communications. 

5.  In-house  or  readily  available  liaison  with  military  personnel  and  sites. 

6.  Reasonable  proximity  to  major  Navy  installations. 

7.  A  history  of  excellence  in  program-related  research. 

These  considerations  suggest  that  the  most  appropriate  research  sites  would  be 
existing  research  organizations  or  independently  managed,  interdisciplinary  re- 
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search  organizations  within  universities.  Most  universities  are  not  designed  to 
undertake  large-scale  development  efforts  on  military  team  performance.  How¬ 
ever,  a  few  have  successfully  undertaken  applied  efforts  either  alone  (e.g.,  Came- 
gie-Mellon  University’s  ARPA-sponsored  speech  understanding  work)  or  in  con¬ 
junction  with  other  research  institutes  (e.g..  University  of  Illinois  with  Bolt,  Bera- 
nek,  &  Newman,  Inc.  in  reading  research;  Carnegie-Mellon  University  with  the 
American  Institutes  for  Research  in  document  design  research). 


6.2.  LABORATORY  FACILITIES 

The  precise  specifications  and  costs  for  the  laboratory  facilities  depend  on  the 
domain  to  be  investigated.  However,  we  can  make  some  general  assumptions  about 
the  required  laboratory  capabilities. 

If  the  task  domain  is  an  information-processing  decisionmaking  task  such  as 
that  performed  in  Navy  CICs,  the  laboratory  must  be  able  to  accommodate  between 
ten  and  twenty  team  members,  including  receivers,  users,  and  integrators  of  infor¬ 
mation,  decisionmakers,  and  communicators.  Each  of  these  team  members  will 
require  some  form  of  display  device,  a  keyboard,  and  a  method  of  communicating 
with  other  team  members.  Communications  will  probably  require  both  telephone 
and  computer  network  links. 

A  central  laboratory  processor  will  be  required  to  perform  several  functions 
during  team  experiments;  maintaining  a  representation  of  the  combat  situations, 
providing  a  source  of  enemy  activities,  generating  appropriate  inputs,  controlling 
the  simulations,  and  maintaining  an  event  memory  for  data  collection  and  subse¬ 
quent  analysis.  In  addition,  this  processor  must  support  high-level  programming 
languages,  word  processing,  ARPANET  interfaces,  and  message-handling. 

Finally,  additional  equipment  will  be  required  to  support  the  observation  of 
team  performance  by  researchers.  Typical  equipment  might  include  a  large  graph¬ 
ics  display  for  all  scenario  activities  and  assorted  recording  equipment  (e.g.,  q 
videotape  recorder  and  camera). 

At  least  three  areas  would  be  required  to  house  this  facility;  a  laboratory  that 
contains  the  display  stations  and  in  which  the  teams  perform,  a  hardware  area  to 
house  the  processors  and  storage  devices,  and  a  conference  area  with  large  projec¬ 
tion  equipment  to  accommodate  experimenters  and  observers.  This  entire  facility 
would  require  about  1,500  to  2,000  square  feet  of  floor  space. 

In  computing  approximate  costs  for  this  laboratory,  we  assumed  that  the  cen¬ 
tral  processor  would  have  computing  power  equivalent  to  that  of  a  Digital  Equip¬ 
ment  Corporation  VAX  11/780  and  that  the  power  of  each  display  station  would 
be  equivalent  to  that  of  a  3-Rivers  PERQ.  On  this  basis,  the  combined  costs  for 
hardware  acquisition,  software  acquisition,  and  systems  support  software  for  the 
laboratory  would  be  approximately  $  1  million.  This  cost  could  be  amortized  over 
the  duration  of  the  research  contract,  and  of  course,  it  would  be  lower  for  institu¬ 
tions  with  some  or  all  of  the  requisite  equipment  already  operational. 


6.3.  THE  RESEARCH  PLAN 

This  section  suggests  a  set  of  time-phased  activities  by  ONR  management  and 
potential  contractors  for  executing  the  proposed  research  plan.  A  number  of  un- 
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known  factors  prevented  us  from  estimating  the  level  of  effort  that  could  be  com¬ 
mitted  to  each  contractor.  These  unknowns  included  the  number  of  contractors  in 
the  program,  the  current  laboratory  capabilities  of  the  contractors,  the  number  of 
research  projects  undertaken  by  each  contractor,  and  the  precise  amount  of  money 
available  in  the  research  program.  Thus,  rather  than  proposing  expected  levels  of 
effort,  we  restricted  ourselves  to  categories  of  activities  to  be  undertaken  at  each 
stage  of  the  program.  We  base  our  plan  on  a  projected  six-year  program,  including 
the  current  fiscal  year  (FY80). 

ONR  should  immediately  undertake  several  activities  to  initiate  the  research 
program: 

1.  Establish  the  budgetary  guidelines  for  the  entire  program. 

2.  Form  the  steering  committee  to  monitor  the  research  (see  Section  4.3). 

3.  Enlist  tri-Service  and  other  agency  support  for  the  program  (see  Section 
4.7). 

4.  Identify  and  select  the  long-term  contractor(s). 

5.  Fund  start-up  efforts  for  the  contractor^)  (see  below). 

6.  Organize  a  workshop  for  the  contractor(s)  and  relevant  operational  and 
training  personnel. 

The  scope  of  the  research  activities  recommended  in  Section  5  is  sufficiently 
broad  that  we  assume  only  a  subset  can  be  undertaken  simultaneously  (unless  there 
were  very  substantial  funding  and  multiple  Research  Centers).  Therefore,  once  the 
individual  contractors  have  been  selected,  they  must  develop  a  detailed  research 
agenda.  While  the  particular  interests  and  capabilities  of  the  individual  contractors 
will  determine  the  specific  proposals,  we  suggest  the  following  research  strategy. 

In  the  current  year,  ONR  should  focus  on  the  selection  and  analysis  of  target 
teams,  the  identification  of  research  requirements,  and  the  design  of  a  research 
laboratory.  The  following  specific  research  activities  (described  in  Section  5)  should 
be  undertaken  because  they  have  immediate  utility  for  the  operational  Navy  and/ 
or  because  they  provide  the  laboratory  requirements  and  design: 

1.  Navy  teams  of  the  future  (Section  5.1.1) 

2.  Development  of  a  task  specification  for  tactical  decisionmaking  (Section 
5.2) 

3.  Cognitive  models  of  individual  performance  (Section  5.4.1) 

The  completion  of  these  research  efforts  will  depend  on  the  capabilities  and 
funding  level  of  the  individual  contractors  and  the  date  on  which  research  is  begun. 

The  second  year  should  continue  first-year  research  and  expand  into  a  larger 
set  of  activities.  The  goal  of  the  second  year  would  be  to  order  and  install  available 
laboratory  equipment  and  use  the  results  of  first-year  research  to  design  one  or 
more  pilot  experiments.  The  pilot  experiments  would  provide  a  shakedown  of  the 
available  laboratory  facilities,  assist  in  the  development  of  task  scenarios,  and 
provide  some  initial  results  of  team  experiments.  The  candidates  for  second-year 
research  include  the  following  activities: 

1.  Current  technologies  for  improving  Navy  teams  (Section  5.1.3). 

2.  Environmental  simulations  for  mock  operational  equipment  (Section 
5.3.1) 
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3.  Simulation  and  gaming  scenario  development  (Section  5.3.2) 

4.  Investigation  of  the  relationship  between  individual  and  team  perfor¬ 
mance  (Section  5.4.3) 

5.  The  dynamics  of  team  evolution  (Section  5.5.1) 

6.  Enriching  feedback  in  training  and  operational  environments  (Section 
5.7.6). 

The  outcome  of  these  research  activities  should  lead  to  the  design  of  larger- 
scale  experimentation  and  the  selection  of  subsequent  high-priority  research  topics. 

The  third  year  would  see  the  completion  of  the  laboratory  installation  and 
debugging.  During  this  year,  the  research  program  would  focus  on  producing  the 
first  major  research  results.  It  would  also  expand  to  include  additional  research 
activities  as  dictated  by  laboratory  availability  and  early  research  results.  The 
promising  candidates  for  this  stage  of  the  research  include  the  following: 

1.  Organizational  factors  in  introducing  innovation  (Section  5.1.2) 

2.  Capabilities  for  distributed  exercises  (Section  5.3.3) 

3.  Theories  of  team  performance  (Section  5.4.4) 

4.  Investigation  of  alternative  decisionmaking  strategies  (Section  5.6.2) 

5.  "Intelligent”  computer-assisted  instruction  for  team  training  (Section 
5.7.1) 

6.  Training  flexibility  in  SOP-based  performance  (Section  5.7.4) 

7.  Automated  planning  aids  for  decisionmaking  (Section  5.8.2) 

8.  Machine  aids  for  cooperative  problem-solving  (Section  5.8.3) 

At  the  end  of  the  third  year  or  during  the  following  year,  ONR  should  decide 
how  to  continue  the  research  program.  In  particular,  it  should  determine  whether 
to  continue  with  the  same  or  an  altered  number  of  contractors,  what  research  areas 
seem  the  most  promising  (based  on  the  results  of  policy  studies  and  laboratory 
development,  and  initial  research),  and  how  successful  the  contractors  have  been 
at  meeting  research  objectives.  Possible  decisions  include: 

1.  Continue  funding  of  all  contractors  to  perform  additional  research. 

2.  Continue  and  possibly  increase  funding  of  fewer  contractors. 

3.  Delay  the  decision  because  no  contractor  has  assembled  the  desired  facili¬ 
ties. 

4.  Redirect  the  program  to  smaller  efforts  either  more  removed  from  specific 
applications  or  focused  on  a  smaller  set  of  research  issues,  as  in  current 
ONR  6.1  programs. 

5.  Terminate  the  program. 

Assuming  a  positive  decision,  research  would  then  continue  for  three  to  seven 
years  (to  a  total  of  ten  years),  subject  to  negotiations  between  the  contractors  and 
ONR.  Candidate  areas  for  new  research  initiatives  include: 

1.  Causes  and  effects  of  Navy  team  turnover  (Section  5.1.4) 

2.  Team  members’  mental  models  for  task  performance  (Section  5.4.2) 

3.  The  effects  of  team  turnover  on  performance  (Section  5.5.2) 

4.  Task  restructuring  (Section  5.6.1) 

5.  Investigation  of  alternative  communications  structures  (Section  5.6.3) 

6.  Training  mental  models  (Section  5.7.2) 
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7.  Training  empathetic  models  (Section  5.7.3) 

8.  Dynamic  gaming  as  a  training  aid  (Section  5.7.5) 

9.  Automated  specialists  to  supplement  or  replace  team  members  (Section 
5.8.1) 

The  suggested  time  scale  and  research  priorities  listed  above  are  intended  oaly 
as  guides.  As  indicated  earlier,  the  funding  level  and  capabilities  of  the  individual 
contractor(s)  will  be  significant  determiners  of  the  extent  of  the  program  and  the 
order  in  which  research  activities  are  undertaken.  The  recommended  development 
sequence  should  lay  the  foundation  for  subsequent  and  more  extensive  research  on 
the  specific  team  improvement  research  efforts  suggested  for  the  third  year  and 
beyond.  The  establishment  of  several  Research  Centers  with  particular  capabilities 
may  lead  to  an  overlap  of  effort  on  some  research  activities,  and  to  a  division  of 
effort  on  others.  Some  duplication  of  effort  should  be  anticipated  and,  in  some  cases, 
encouraged,  particularly  for  experiments  in  which  the  generality  of  the  research 
results  is  important.  As  research  results  become  available,  ONR  and  the  contrac¬ 
tors  should  work  with  the  applied  development  and  operational  communities  to 
create  Navy  testbeds  for  these  results.  At  the  conclusion  of  the  research  program, 
the  contractors  should  deliver  the  products  discussed  in  Section  2.6. 
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